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toward a total career-development program; the need for functional, 
job-related literacy training; the need to consider oral skills as 
well as literacy skills; and the need for more basic research. 
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for future R&D in the Services. The major recommendations stressed 
the following needs: 

(1) Need for each Service to develop a comprehensive plan leading 
toward a total career development program • 

(2) Need for functional, job-related literacy training. 

(3) Need to consider oracy skills as well as literacy skills. 

(4) Need for more basic research in the areas of reading, read- 
ability, comprehensibility, and useability • 
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SUMMARY AND RECOMMENDATIONS 



j^^jlLITARY PROBLEM 

Whenever the job or training performance of military personnel is 

adversely affected by inadequate literacy skills, the Armed Services face 

a literacy problem. The Services cannot rely upon the nonacceptance or 
limited assignments of marginal literates to avoid this problem. Times 
of increased mobilization or lower economic opportunity will bring a 
certain percentage of marginal literates into the Services, as will 
considerations of equal opportunity and upward mobility. Furthermore, 
the increasingly technical nature of military documentation dejnands high 
. literacy skill levels; even personnel of moderate reading ability may fall 
short of these high levels. 

The overall "literacy problem," which involves both oral and written 
language skills (oracy and literacy), has two sides: (1) the personnel 
side - problems of assessment of the literacy and oracy skills of personnel 
and of attempts to raise those skills through literacy training; and (2) 
the materials side- problems of assessment of the reading difficulty 
(readability, comprehensibility , useability) of job and training materials 
and of attempts to lox^zer the difficulty of those materials and to other- 
wise modify job and training requirements to reduce their literacy demands. 

RESEARCH AND DEVELOPMENT PROBLEM 

The Armed Services have active research and development efforts which 
address both sides of the literacy problem. Considerable R&D activity on 
this problem also exists outside the military. Increased interaction 
between Armed Services and civilian R&D personnel would provide opportunity 
for fresh insight and ideas for future R&D activities within the Services, 
To meet such an objective, an agreement was made to hold a Conference which 
would bring together key military and civilian personnel in these R&D areas. 

APPROACH 

A Conference on Reading and Readability Research in the Armed Services 
provided an opportunity for an exchange of ideas between military R&D 
personnel and civilian consultants. In this interchange, past and present 
Armed Services R&D efforts, delivered in perspective papers, were discussed 
in Working Group sessions which met to consider recommendations for future 
R&D. Historical papers were delivered by J, Dexter Fletcher (NPRDC) , 
Jack Hiller (ARI) , and James R, Burkett (AFHRL) . Papers on current 
research on the personnel side of the literacy problem were delivered 
by Thomas Duffy (NPRDC), John Caylor (HumRRO) , and Steven Groff (AFHRL). 
Papers on the materials side of the literacy problem were prepared by 
Thomas Curran (NPRDC), Arthur Siegel (Applied Psychological Services), 
Richard Kern (ARI) , Robert Johnson (AFHRL), and William Muller (NAVAIRSYSCOM) . 
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The civilian consultants, who led the Working Group sessions, were 
Jeanne Chall (Harvard), Samuel Gibbon (Children's Television Workshop), 
Robert Glaser (LRDC) , John Guthrie (IRA), George Klare (Ohio University), 
Michael Macdonald-Ross (Open University), Ernst Rothkopf (Bell Labs), 
and Richard Venezky (University of Wisconsin). Macdonald-Ross presented 
the Conference, keynote address, "Research and the Transformer," which 
gives a unique perspective to R&D efforts. This X'olume presents a record 
of those Conference activities. 

MAJOR RECOMMENDATIONS 

While the Conference generated many specific recommendations and 
suggestions for future R&D, these recommendations may be grouped into 
four main concerns. 

(1) The need for each Service to develop a comprehensive plan 
leading toward a total career development program. Such a program would 
incorporate R&D efforts on both the personnel and the material sides of 
the literacy problem and would provide the individual with access to 
literacy training and job training, Xv^hen needed, throughout his career 
development. 

(2) The need for functional, j ob*--r elated literacy training • Unlike 
general literacy training, functional literacy trainin,'^ can integrate into 
job training and on-the-job e:':periences , providing motivation in terms of 
successful job^related performance. 

(3) Tne need to coasider oracy skills as well as literacy skills. 
Oral language skills can affect the nature and amount of literacy training, 
and also job training, that will be most effective for an individual. 

Much research on oracy skills remains to be done. 

(4) The need for more basic research in the areas of reading, 
readability, comprehensibility , and useability. Such research would 
increase our knox^;ledge of the processes involved in reading and reading 
training; the increased understanding would, in turn, help us to produce 
both better readers and better reading materials. 
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FOREWORD 



In October :975 a ConTerence on Reading and ReadabiVity Research 
in the^Ar.Tic-d Sei-.-i ces v;cr;; held at the U.S. Naval Postgraduate School 
in Monterey, Ca 1 iforma . Thi' •Conference brought together representa- 
tives from research and developiT;ent organizations wUhin the U.S. Air 
Force, Arny, and Navy, cpf rational personnel from these services, and 
civvlian scientists from major universities and other research and 
aevelopment organisations to discuss mutual concerns on issuer; of read- 
ing assessment and training, and the design of more readable and use- 
able technical docun^entatior, . This reoort presents the proce^^dinos of 
tne Conference and additional papers resulting from Dosc-Conference 
del tberati one . 

The Conference Wds f-ndsd by all three a)-med services. Specific 
. organizatTcns, and p-onle ;n those organizations whose activ^:ties made 
tne Conference possibi..; 

Air Force Office o^' Si:ientif ir. Research, Dr, Alfred R. Fregiy 
Air Force Hunac; Resjurcas Laboratory, Dr. dames R. Burkett 
Army Research [nht^tute 

for the Behavioral v/' 3&ciai Sciences, Dr. J. E. Uhlaiier 

Navy Personnel Rssr:r.'v.:- and Development Center, Dr. Edwin Aiken 

Office of naval --.os^i-zh, r.r. Marshall J. Farr 

In addition to tho :"::r-:'qo-:nq persons, Dr. Joseph Hard o^^ Arny 
Research Institute for zh^: Behavioral and" Social Sciences contributed 
considerably to the de^.'elopment of interest in and support for the 
Conference. 

^Jhe task of serving ds technical monitor for the project fell to 
Dr. Marshall J. r^rr of the Office of Naval Research. In this role he 
served as the primary contact for HumRRO personnel, coordinated contract- 
ing and Conference planning meetings among the various service organiza- 
tions, and provider counsel ^ind assistance on various matters nertain- 
ing to the conduct and reporting of the Conference. 

^ Rear Admiral Ishnni Linder, Superintendent, U.S. Naval Postgraduate 
ochooi, kindly mads the facilities of that institution available for the 
Conference. Captain Dean Taylor, Jr/, Chief of Staff of the USNPGS, 
acted as host and provided support for the Conference. Mrs. Ruth Guthrie, 
Conference Coordinator at the USNPGS, performed valuable ■ services in the' 
arrangement of facilif'es and services for the Conference. 

The Conference was conducted by the Human Resources Research Organi- 
zation, Western Division, where Dr. Howard H. McFann is Director. Dr. 
Thomas G.^Sticht was Princ-ipal Investigator for the project. Mrs. Diana 
Woity Zapf served as Conference Coordinator and Associate Editor of the 
Conference proceedings. 

The work was- performed under Department of the Navy Contract N00014- 
/6-C-0312 issued by the Office of Naval Research under Contract Authority 
NR 154-385, with funding supoort supplied by the Office of i^aval Research, 
the Navy Personnel Research and Development Center, the Army R^seo-^'h 
Institute for the behavioral and Social Sciences, and the Air Forr-^' Of^^:f:p 
of Scientific Research. 
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THE CONFERENCE ON READING & READABILITY RESEARCH 
IN THE ARMED SERVICES 



BACKGROUND TO THE CONFERENCE 

Tha Literacy Problem: Producing Better Readers 

In the midst of Project 100,000 in November of 1970, a Wo^xlUng G^.oup 
on L^tzyicng and K^acLuig ui ihz kmzd SeJivlcU convened at the Presidio 
of Monterey, California. The purpose of this Working Group was to study 
ths literacy problems created for the Armed Services- by the large influx 
of less literate men of Project 100,000. 

As a consequence of its deliberations, the Working Group prepared 
a recommendation for literacy research and development in the Armed Serv- 
ices and submitted this recommendation to Dr. Ralph Canter, then Director 
of Research, Office of the Assistant Secretary of Defense, Manpower and 
Reserve Affairs. 

In April of 1974, the Manpower Development Division of the Air Force 
Human Resources Laboratory released a RtpO^t Q n L itOAacy VicUiving Jho- 
g^iom tixd K^qA S^AvIc^1> (McGoff & Harding, 1973). This report con- 
tains a Prologue signed by the then Deputy Assistant Secretary of Defense 
for Education, Mr. M. Richard Rose, who comments on the problem of lit:- 
eracy as it continues beyond Project 100,000 and into the All Volunteer 
Force. This Prologue states: 

It appears hi^'/.ily probable that the estimates of a con- 
tinued flov of ar..--ssions with reading problems into the Armed 
Forces are ess«-. ^ '..;lly correct . . . 

The need for a more comprehensive effort on this problem 
is evident in the unanimous recommendations of the Working 
Group on Listening and Reading in the Armed Services made in 
November 1970, This recommendation was that 'literacy train- 
ing be designed following a system approach, which would include 
the thorough assessment of literacy requirements of the various 
military occupations, the orderly structuring of training pro- 
grams geared to satisfying the occupational requirements, and, 
mo/^.t AJmponXxDfVtly , well designed evaluative procedures to pro- 
vide feedback for program improvement' < . . 

Progress. in dealing with the literacy problem is essential 
if the productive potential and more effective utilization of 
. a portion. of our manpower resources are to be achieved. , 

(McGoff & Harding, 1973, pp 9-10.) 

As this quotation indicates, as', of April 1974, there was continued 
concern within the Armed Services with training problems, posed by the need 
to utilize personnel haying low literacy skills. Since then, the economy 
has stimulated many more academically capable youth to seek empla>Tnent in 
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the Armed Services, and the problem of extremely large nu:.ibers of margin- 
ally literate personnel has at least receded, although it has not dis- 
appeared conpletely. Furthermore, it seems likely that if economic con- 
ditions in the nation were to improve considerably, the Services would 
cnce again face the problem of providing literacy training to accommodate 
less apt personnel. 

Facing the .foregoing problem directly, all three Services - Army, 
Navy, Air Force — today are vigorously researching methods for provid- 
ing reading training in an optimal manner for career development of all 
personnel. Tliis means that, while attei;,t.lon is being given to the very 
lowest level of reader who is likely to enter the Service, there is also 
concern with the development of reading training for personnel reading 
anywhere below the high school level. Many of the higher quality recruits 
entering the Services today under the press of economic insecuiitv are 
reading well below the high school level for which they are certified by 
-diploma. Hence, even with the higher quality input into the 5>cr^-icet^ 
today, there is a continued need for research and development on methods 
for providing literacy ^:raining for career development . .-Vnd, indeed, 
such research is underway in all three Services. 

The Literacy Problem: Producing Better Reading Material ^ 

A m.ajor problem facing laany military personnel i^ th:.i'c the technical 
manuals, training literature, job performance aids, and other \rritten 
m-aterials that they are given are frequently of such poor quality (high 
difficulty, poor format, incomplete, poorly organi;:ed) a^ to render them 
practically unuseable , -^r egardless of a person's readin- ability. Again, 
this problem has been well recogniz-d by the three Services, and active* 
research and development programs are currently underway to improve tech- • 
nical documentation. The success of these R&D efforts should le.ad to re- 
duced reading demands of technical materials in many military jobs, and • 
hence, render such jobs more accessible to less literate personnel (as 
v/ell as producing materials more useable by all personnel). 

The R&D Problem 

Thus, the Armed Services have research and developm.ent programs which 
take two main approaches to literacy problems: RriD to p/ioducd b?XtCJL 
n.za.dojjj, through literacy training, and R&D to p/iodiiCP. bojUd^l JidadLng 
maX-CJiMs^ i.e., to reduce the difficulty levels of manuals, texts, and 
other printed materials. However, in recent conversations with scientists 
in each of the Services, it was acknowledged that there is currently a 
great deal of research and development activity outside the Armed Services 
on problems of literacy and design of textual materials which has relevance 
to the problems found in the Armed Services. A need was expressed by re- 
searchers within the Department of Defense to communicate with experts on 



In lliLs report, we uf:-:e iho. spelling it^zabWAXtJ rather rh;-i U."^ rd) UlLtf f . We 
heiiifvc* that th(i former impelling h(?.Lps preserve the soniinLic re.laiiioii 
u^it\'Ji\r'n.LU^il and CJlbll.iL tj , a relat i^on of major J.mportanr.i- in LmDroving 
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literacy and readability from civilian institutions. It was felt that 
such discussions might provide fresh, creative insights for long-term 
solutions to the recalcitrant problems of literacy and imprcvement of 
technical writing which the Services face, 

A Conference on Reading and Readability Research in the Anned Services 

Because of the expressed desire by many Armed Service R&D personnel 
to meet ivath and interact with civilian literacy experts, as well as to 
interact with each other in response to conversations with civilian ex- 
perts, the Human Resources Research Organization (HumRRO) proposed to con- 
duct a Coni^eAe^ice on Rzadaig and RzadabltiXy Kuzcuick In ;t}iz Amzd SeAvlcu, 
Such a Conference would bring together Armed Services R&D personnel, con- 
cerned military operational personnel, and civilian experts in literacy 
and technical writing to focus specifically upon literacy and technical 
^vrriting R&D problems within the Armed Services, 

This proposal was discussed x^^ith representatives of the. Office of 
Naval Research, the Navy Personnel Research and Development Center, the 
Air Force Human Reso'urces Laboratory/Tecunical Training Division, and the 
Army Research Institute for the Behavioral and Social Sciencey. 'Durine 
these meetings, basic agreement on the need for such a Conference was ^ 
reached, and it was also suggested that, following the Conference, a work- 
ing group composed of representatives from the R&D organisations of the 
Navy, Army, and Air Force and tlic Conference Chairman from HumKRO should 
meet to consider the proceedings of the Conference and to study recommen- 
dations for R&D in reading and readability from the Conference for inte- 
gration into the Conference Proceedings for wide-scale dissemination, 

^As a consequence of Lhe negotiation meetings with the four R&D organ- 
izations from the Armed Services, the Office of Naval Research, Personnel 
and Training Research Programs Division, was authorized to act on behalf 
of the three Services and to contract with HumRRO to conduct the Con{^^lzncZ 
on Reading and RzadabltUij Ruzcuicli .ui thz hnnzd S2Avlacyb. 

OBJECTIVES AND FORMAT OF THE CONFERENCE 

Major Objectives of the Conference 

The i:;oal of the Conference, that of exchanging ideas in consideration 
of the course of reading and readability R&D, is embodied in these major 
ob j ec tiveH : 

1. To bring together military R&D and operational personnel with 
civilian experts in the fields of reading and readability of technical 
writing to discuss the Services' problems in these areas"; to discuss 
what the Services have done, what they are currently doing, and what 
each Service is anticipating for immediate next steps in these areas; 
and to exchange ideas and insights for additional actions which might 
be taken to pursue solutions to the problems of literacy and technical 
writing encountered by the Services. 

15 
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2. To develop, through snail-group discussions of key issues and 
i^ea papers produced by the civilian expprts based upon their reflec- 
tions of the Conference, information which would be reviewed in post- 
Conference, tri-Ser^/ico meetings to develop r ecoTnniendations for future 
K5iD in reading and readability. 

3. To produce a published document containing the proceedings 
of the Conference, including the recommendations evaluated in the 
tri-Service, post-Conf ersnce meetings, which would bring the Armed 
Services' concerns with literacy and readability of technical writ- 
ing to the attention of a Xv^ide range of Department of Defense per- 
sonnel and civilians interested in these vital issues- 

Format of the Conference 

The Confcpcncp. on Reading and Readal)ility Reseca'ch {.k vhe Armed 
SemHces was held on October 28-30, 1975, at the Naval Post-graduate School 
in Monterey, California. The mornings were devoted to presentations of 
perspective papers prepared by research personnel for and from the Air 
Force, Army, and Navy.. The papers dealt with (1) pa^t liL]} on literacy 
training and readability, (2) oyigoing. reseunrch on literacy training, (3) 
ongoing research on improving the readability of technical vnriLing, and 
(4) ongoivxj research on .Improving the useability of technicil ^-I'iting 
through '^uHsideration oL Factors other than tho^je involved in readability 
research (e.g., fonnaL-, organisation, selection of conr:ent via tasl: anal- 
ysis). Fallowing each paper, a non-Service related civilian e:<pert in 
reading or technical documentation (readability, design of texts) commented 
on the paper. These experts served as consultants i:o the Conference and 
x/ere available to Service representatives throughout the Conference. The 
perspective papers and the comments by the civilian consnlLanLs' are con- 
tained in ParL II of Lhese Proceedings. 

In the afternoon sessions, the expert consultants to the Conference 
conducted five Working Groups dealing v/ith these RaD topics: (1) reada- 
bility, (2) design of texts and graphic material, (3) training systems 
and materials, (A) literacy training programs, and (5) basic skills in 
reading (phonics, comprehension). These Working Group sessions discussed 
various issues in these topic areas and suggested recommendations for 
future research. 

At a general meeting of conferees on the last afternoon of the Con- 
ference, The Working Group leaders (the civilian cpnsulLnpcs) presented 
oral summaries of; the key issues and recommendations to '..imerge in their 
groups. At this same general meeting, operational personnel from each 
of the three Services delivered oral statements of resc.irch and develop- 
ment needs from an operational perspective. The combined observations 
and suggestions that emerged from the Conference are presented in Part III 
of these Proceedings. 

In addition to the papers in Parts II and III, tvo p/irr^:,-,- wc-c^ prepared 
whicli present perspec. t ivi^s on literacy training and design of materials that 
cut across all three Stirvices, Tliese papers follow in the reinainin:.3 pages 
of Part I. 

16 
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LITE-^'CY TRmim IN THE ARMED SERVICE 

Thomas G. Sticht 
Human Resources Research Organization 
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STRATEGIES FOR DEALING WITH LITERACY PROBLEMS 

Four strategies have been followed at various times to cope with the 
problems of illiteracy (or marginal literacy) in the Armed Services. Two 
of these strategies, selection and classification, depend for their sue-- 
cess upon adequate assessment instruments for predicting who will be suc- 
cessful or unsuccessful on the nob. The remaining tvo strategies involve 
the development of training programs for *;lower cTptitude, less literate 
personnel. In one approach, the person's low literacy skills are consid- 
ered as a fixed characteristic of the person, and modification of\job 
technical training programs adapt to the person's lower^than-noirmal 
capabilities is attempted. In :'ie fourth str-j.tegy, an attempt is made to 
produce a more adaptive person through the provision basic literacy 
skills training. 

Nonacceptance of Illiterates 

One way to overcome the problems resulting from low literacy skills 
in inductees is to avoid them. From time to time, the Services have 
raised admittance standards with the result that large numbers of mar- 
ginally literate men were excluded from service. In general , standards 
have been raised during intervals of rf-lative military quiescence and 
lowered during periods of military activity, such as the Korean and 
Vietnam conflicts. 

There are, however, several problems associated with the strategy 
of nonacceptance that limit its f ruitf ulness. For one thing, as with 
many other abilities, it is not a simple matter to accurately assess a 
man's literacy skills at the selection station. Hence, large numbers of 
potentially useful men may be turned away, while some who are not useful 
nay be accepted. 

The problem of accurately selecting men on the basis of their liter- 
acy skill is compounded by the fact that, until the research reported by 
Caylor in this volume, no attempt had been made to accurately identify 
literacy skill levels required by military jobs and training schools. 
Therefore, there were no adequate criteria for selecting cutoff points 
on selection tests. 

Both of the foregoing problems are concerned with assessment — the 
first \'j±th assessing the man, and the second with assessing the job re- 
quirements. A third problem affecting the usefulness and desirability of 
the strategy of nonacceptance concerns training. Manpower needs are such 
thac it may become necessary, under conditions of large-scale mobilization, 
to enlist marginally literate men. If these men are not accepted during 
peacetime and the training methods needed to keep pace with technological 
change developed to effectively train such men, then new training tech- 
niques and methods will have to be developed under the stress of mobil- 
ization, when expediency rather than effectiveness may be the predominant 
training motive. 
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Perhaps the most significant results of the nonacceptance into the 

military of marginally literate people is that a large and needy segment j 

of the population is not able to reap the benefits of the training, edu- j 

cation, social development, and practical experience that accompany mill- j 

tary service, j 

Limited Assignments 

The problem of using marginal literates also has been dealt with by 
assigning these people to jobs that have "minimal" requirements for read- 
ing. This strategy has not worked well for several reasons. First, as 
with the other strategies reviewed, adequate definitions of the require- 
ments for reading skills in different jobs have not been available; hence^ 
it has not been possible to accurately state "minimal" requirements for 
reading skills. 

A second difficulty is concerned with selecting job proficiency lev- 
els for establishing reading requirements. Individuals qualified for 
entry-level jobs are not necessarily qualified for advanced level jobs. 
This may be a particular problem for marginal readers because higher level 
positions tend to place higher demands on reading skills. In work for the 
Nav>', Sticht et al. (1976) found that the amount of time spent in job- 
related reading increased as a function of rank, with pay grades E1-E3 
reading 0,7 hours; E4-E5 reading 1.7 hours; and E6-E7 reading 2.1 hours 
per day. Similar findings were reported in a Department of the Army (1968) I 
survey of the primary reasons for reading among Army officers and enlisted j 
men. For enlisted men, the percentage of job-related reading gradually j 
increased from 2% for Els, to 25% for E8s, and then dropped to 16% for E9s. 
Thus, higher levels of pay and responsibility seem to require a greater i 
amount of reading. To assign personnel to nonreading "tracks" at the I 
start of their careers is to place a definite limitation on their career 
development and restrict their range of utility (say, through cross- j 
training or re-training as new jobs open up) to the Services. j 

A third problem with the limited assignment approach to dealing with 
personnel of low literacy skills is that in most instances, advanced-level ■ 
positions are filled with personnel from the entry-level jobs. The assign- i 
ment cf a man with the marginal requirements needed for an entry-level job 
may result, perhaps because of combat casualties, in his promotion to a 
leadership position, with possible devastating effects for him and the men 
he leads. 

Another difficulty with the policy of assigning the marginally lit- 
erate to a job having relatively low requirements for literacy and arith- 
metic skills concerns the overall effectiveness of that entire job field. 
This may be reduced if the job becomes flooded with marginals. Therefore, 
some means are necessary for distributing these people equitably among 
j obs . 
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M ethodological Considerations in Determining; Reading Demands of Jobs 

The strategies of nonacceptance and limited assignments for marginally 
literate personnel both require, for their most effective implementations, 
infonaation about the reading demands of jobs. Questions such as "What are 
tha reading requirements of different jobs?" "How well does a person have 
to read to be able to do the job?" "What jobs can less literate personnel 
perfortn?** all require, for their answers, information about the reading 
decnands of jobs. However, as the following review reveals, the determina- 
tion of the reading demands of jobs is not a straightforward matter. 

Review of literature indicates that reading demands of civilian and 
military jobs have been stated in two ways. In one way, reading demands 
are seated in terms of the types of reading materials and reading tasks 
the job involves. In a second approach, reading demands of jobs have been 
expressed as a single index number ~ the reading grade level of general 
reading ability needed to perform well on the job. The first approach can 
be called the AmmaAtJ toMk ^totrnZYVt method of stating reading demands of 
jobs. The second approach can be called the ^umnoAy A^ndzx nimih^A method 
of stating reading demands of jobs. 

Estimates of Job Readino; Demands Given in Summary Task Statements 

In tiiiis approach, job analysts or others usually interview management 
personnel, and sometimes workers, to determine whether or not a given job 
requires the use of reading materials. If so, then a simple statement to 
that effect is recorded in the job requirements. 

As an example of this approach, we can cite the Army Regulation^ 
Bi£x6t(id l^MXa/uj OacupcuUo'AoZ SpzcAaJtUu (AR Reg 611-201, 5 Jan 67). 
In this regulation, reading requirements for the Field Radio Repairman 
are given as: ^'Requires verbal ability to read and understand technical 
material pertaining to maintenance of field radio equipment." The mechan- 
ic (Ground Vehicle Repairman) : "Requires verbal and reasoning ability to 
read and understand technical material pertaining to equipment being main- 
tained ..." The Military Policeman: "Requires verbal ability to inter- 
view witnesses and interrogate suspects, prepare written r^^ports of find- 
ings, and read communications." 

Recent work sponsored by the Navy (Sticht et al., 1976) describes an- 
other method of specifying reading tasks. In this case, a checklist of 
materials is compiled and people indicate which kinds of materials they 
read in their job; e.g., notes, notices, messages, manuals, etc. A similar 
method was used by Sharon (1972) to obtain an indication of what kinds of 
materials the adult population in the Unites States read. WhiJ.e such meth- 
ods more precisely identify ichcit people read than does the foregoing ap- 
proach, they fail to identify lohy people read certain materials. 

A recent project by the Department of Manpower and Immigration of 
Saskatchewan, Canada, has refined the method of summary task statements to 
obtain a more finely grained view of the reading tasks performed in various 
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career fields (Siaith, 1975). In this study, an attempt was made to find 
out both ivhcit fU^ncUi of materials, are read in these jobs (e.g., notes, 
nanios, letters, directions, instructions, policy manuals , procedural man- 
uals) and iChcit AZacUng ta^k^ are performed in these jobs (e.g., read to 
locate ar.d understand the main point or idea, to follow directions, to 
put details in sequence or order, to notice and interpret how facts or 
datails are related, to make comparisons). 

To obtain this information, inter'/lewers at times shewed pictures 
of the general type of material they were talking about. For instance, 
in dt:itf-!r:ra.ning if a given job required the reading of graphs, two graphs 
were shoxv"a as exemplars and interviewees were asked to indicate whether 
or not they read similar graphs in performing their job. 

Tinu.s. this method of deriving summary task statements goes beyond 
the preceding method in that it deals not only with {/)kat must be read in 
greater detail, but on w/xy a person reads in a given job. Additionally, 
the Canadian work has distinguished entry-level reading requirements from 
advanced level requirements by asking respondents to indicate what read- 
xn^i; tasks they performed v/hen they first entered the job, and what read- 
ing tasks they presently perform. 

It is clear that the Canadian method provides more useful informa- 
t.-:.on th:?.n the typical rjummary task statement or list of materials people 
road. Iiowever, even though the Canadian approach far surpasses the typi- 
cal curi-nary task statement in specification of reading tasks, it otill 
^f'^^^ precisely specify reading' tasks. For instance, the question of 
I'jKcJ: rr.ritojr^cil^ job holders read versus \A)hy they read are asked separately, 
wiL^iout reference to one another. Thus, though we learn that, in a given 
jon :i perGon reads policy manuals, and this same person reads to follow 
dirccriops, V7e do not know that the person reads to follow directions in 
P0...J.C3/ manualr>. A more explicit statement of a reading task would include 
both a statement of what was read and why it was read. 

There is also reason to question whether people can rate their per- 
formance of a generic reading task, whan all we can pres.cnt are specific 
cli^^plays (texts, figures, tables) with their specific content and (unspe- 
cified) dim.ensions of complexity, legibility, and so forth. The Canadian 
study presented numerous displays which people used as generic displays, 
even though each display was a "species" of the "genus". But that study 
dzd not have any way of determining the extent to which a genus versus a 
species boyis accounted for people's judgments. The work by Sticht et al. 
(1976) found evidence to suggest that people may not respond to the genus 
aspect of a species display, but rather, to the content of the latter. 

Additionally, responding to information displays with a simple "yes, 
T read things like that" or "no^ I do not read things like that'' does not 
define a .t(X/iri, because to "read" is not defined; though, since figures 
and tables are considered to be "read", a general definition like "extract- 
:'ng information from visual displays" may be necessary. But, extraction 
of information can ;o on at various levels. For instance, one can extract 
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inforr.ation about the kind of type (pica?) used; about the color of the 
ink, etc. Or, one can extract information useful for constructing ideas 
represented by the message encoded in the printed display. If the latter 
xs the goal, then we need to know x^hat type of information is being sought 
for what types of ideas and with what type of given display. 

£iStimat es of Job Reading Demands Given in Summary Index Numbers 

Educator Estimates of Job .Reading Demands : Perhaps the most widely 
used method for stating reading demands of jobs is the reading grade level 
(RGL) index number. This appears to have come about because the need to 
state reading demands of jobs has usually occurred along with the need for 
Ixteracy training programs. For instance, as Hiller indicates in this 
volume, in World War II, the Army was required to establish literacy train- 
ing for many illiterate personnel. It was decided that the goal of such 
r.rainiug should be to produce skill in reading up to the level typically 
achieved by children in the fourth grade. This level was established by 
educators as a "gues timate'^ of the minimal reading demands of Army life. 
Since WW II,. whenever large mobilization efforts have resulted in the 
induction of large numbers of marginally literate personnel, such as with 
P:e'ojecl: 100,000 in the nid-sixties, the Air Force, Army, and Navy have all 
established reading training programs having an RGL index number as the 
goal of the program (cf, the papers of Part II of this volume). In Pro- 
.iect 100,000/ the Air Force established a goal of grade 6 reading ability 
for itr, reading program, the Army set its goal at a grade S level, and the 
Nav-y^ prcduG^ed graduates in the 5.0 to 5.5 RGL range. No documentation of 
tlui bar.es^/for these estimates has been found. 

^^'"^ding Demands Derived from Job Analysts' Judgme nts: In civilian 
3-::cting^, the Department of Labor (DOL) has sought ways to establish the 
reading /demands of jobs, again for the purpose of providing guidance on 
objecti,ves for programs for preparing marginally literate persons x^ith 
markotab.le reading skills. 

In the DOL approach, job analysts estimate the levels of General Edu- 
cational Development (GED) required for various jobs, based on interviews 
with job incumbents, supervisors, and observation of the job being per- 
fcnned. Jobs are then categorized as requiring one of six levels of GED. 
iliese levels have been developed to roughly parallel school-based educa- 
tional development. Hence, for example, a GED of level 1, is said to 
approximate the education obtained in grades 1 through 3, level 2 parallels 
4 through 6th grade education, etc. (Phillips, 1970). Thus, when a job 
is assigned a GED level, it has also been assigned a reading grade level. 
To say a job has a level 2 GED, is to say it requires a 4th- to 6th-grade 
reading ability- 



22 



21 



This approach, like that of th : educator's estimate, is a jvdgmQyvtaZ 
approach, v;hich calls for an-estimate by the job analyst. Though rela-- 
tively lox7 in cost, the lack of specificity in the rules for arriving at 
adjudgment of the GED level, and hence reading level of the job, and the 
absence of empirical information which validate the estimates of the job 
analyst (or educator) renders this method too imprecise and uncertain for 
establishing reading demands of jobs. 

Readabil ity Estimates of Job Reading Demands : The development of^ 
methods for estimating the reading grade level of difficulty of various 
• reading materials, by specially constructed readability formulas, makes 
possible a relatively low cost method for estimating reading demands of 
jobs. By applying a readability formula to samples of job reading mate- 
rials, nn average reading grade level of difficulty for the materials can 
be computed, and used to represent the reading requirements of the job or 
job training program. 

Readability formulas have typically been constructed by t\jo means. 
In one approach, prose passages from school textbooks of various grade 
levels are sampled and features such as average sentence length, number 
of one-syllable words in 100 v/ords, and so forth, are determined. These 
reatures are then used in correlational analyses to find out how well 
tney^can be used to predict the school grade level of material. Generally, 
one finds that averagf^ sentence length, and word length increases as mate- 
rials trom higher grades are sampled. Because of this positive correla- 
tion, it is possible to obtain a sample of material, determine average 
values for features such as sentence and word length, enter these values 
xn a regression equation, and then state that materials having, those val- 
ues are typically found, say, in the 6th grade of school. Therefore, the 
material is said to be of 6th-grade difficulty, and by definition, re- 
quires 6th-grade reading skills- 

The preceding approach to the construction of readability formulas 
coes not involve any direct measure of people's abilities to comprehend 
the materials. It is not kno\^ in that case whether 1/3, 1/2, 1/4, etc. 
of the 6th-grade students can actually comprehend the material having the 
structural features typical of materials found" at the 6th grade. For 
this reason, most readability formulas have been constructed to relate 
leatures of textual material such as sentence and word length to perform- 
ance on tests of comprehension of the material. Then a reading grade 
level ir. assigned to the material by setting a criterion of accuracy on 
the comprehension test, say 70% correct, am^ determining the earliesc 
grade level at which some designated proportion of people, say 50%, get 
70% correct on the comprehension test. If it happened that, on a given 
comprehension test, it was not until the 6th grade that 50% of the stu- 
dents got 70% correct, then that material would be assigned a reading 
difficulty or readability value of 6th grade. 
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To use such a formula for determining the reading demands of a job, 
one must: 

!• Identify job reading materials. 

2. . Sample the materials representatively. 

3. Calculate the critical features (e.g., average sentence length). 

4. Use these features in the readability formula to obtain an 
estimate of the comprehension score 70% of the people would 
get if they were to take a comprehension test like the one 
used to construct the formula. 

5. Convert that score to a grade level score using appropriate 
tables. (If step 4 provides a direct estimate of .reading diffi- 
culty in reading grade score equivalents, then step 5 is un- 
necessary. ) 

6. Obtain the average reading grade level of all the materials 
sampled. This is the reading level demanded by the job. 

Clearly, the foregoing method for determining reading demands of jobs 
is a function not only of the features of the text materials, but also of 
the comprehension test items, the criterion set as acceptable on the com- 
prehension test (e.g., 70% correct), and the criterion set for the propor- 
tion of people at each grade level (e.g., 50%) who must. achieve the cri- 
terion score. As an example, in a study for the Army (Caylor, Sticht, 
Fox, £t Ford, 1973) changing the criterion score on the comprehension test 
from^30% correct to 35% correct, a 5% increase, changed the reading grade 
level assigned to some materials by as much as 2 to 3 grade levels! 

Beyond these problems inherent in the methodology for developing 
readability formulas, there are additional problems in using this approach 
for determining reading requirements of jobs. For one thing, it may not 
be possible to obtain a representative sample of job materials, nor even 
to determine ^ the proper domain of materials from which sampling should be 
done. In this regard, a major difficulty can arise due to the distinction 
between the {^OHmoZ job toAk ^p^(ll{^lcxuUom and- the actual, A.n^OAmal job 
toMU which are performed on a day-to-day basis. The problem is that, if 
supervisors, management, or content experts are consulted to find out what 
reading materials a person must use in doing , the job, they are likely to 
base their statements on their conception of the f^omat, or even idealized, 
job and prepare a list of materials which no one could reasonably be ex- 
pected to encompass, and which are in fact not used in the work-day acti- ' 
vities. 

Interviews with employees may also produce a distorted sample of job 
reading materials if employees tend to "fake good" in response to how much 
and what it is they read. On the other hand, experienced employees may 
fail to report certain reading materials which they used in first entering 
the job, but no longer use. It is necessary, therefore, to ensure that 
new and old employees are interviewed in identifying job reading materials. 
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A second problem with readability measures is that they tend to set 
reading requirements somewhat' higher than do the other empirical methods 
to be discussed below. Work by Caylor, et aL (1973) showed that some 
Army jobs would require 12th-grade or higher reading skills if readability 
factors alone were considered. But because many persons with reading 
skills well below the 12th -grade level were successfully completing car- 
eer education programs and performing successfully on the job, we must 
regard the reading requirements suggested by the readability analysis 
With some caution. 

Estimates of Reading Demands of Jobs Stated in Index Numbers Derived 
from Correlating Reading and Job Proficiency Measures : A general method 
for estimating job reading requirements is the traditional psychometric 
procedure used for validating selection and classification tests. In this 
procedure, performance on a reading predictor test is related via corre- 
lational techniques to performance on' a job proficiency test. If a suf- 
ficiently high relationship exists, cut-off scores on the reading predict- 
or variable can be selected to increase the probability of obtaining stu- 
dents or employees who ^d.11 reach an acceptable level of achievement on 
the job proficiency criterion measures. — 

In research for the U.S. Army (Caylor, this volume) this psychometric 
model was .applied to determine reading demands of four jobs: Cook, Mech- 
anic, Supply Clerk, and Armor Crexraan. Some 400 men experienced in each 
job were administered standardized reading tests and two measures of job 
proficiency: a 4- to 5-hour individually administered job sample test, 
in which men performed actual job tasks derived from extensive job and 
task analyses; and a job knowledge, paper-and-pencil test designed to 
include questions about information actually needed to do the jobs. 

To establish reading demands of these jobs, it was necessary to de- 
velop decision rules for determining what level of reading skill was asso- 
ciated with a desirable level of proficiency in the job sample and job 
knowledge tests. The decision rule finally arrived at stated that the 
lowest level of reading which should be used to establish the reading de- 
mands of the job is the level at which men would not be expected to be 
overrepresented in the bottom quartile of performers. In this case, the 
criterion of job proficiency was a relative, one, being defined in terms 
of quartiles. It was judged that if a person was performing at a level 
below 75% of his fellow job incumbents, and if this was systematically 
related to reading skill, then it would not be overly demanding to target 
the objectives of a reading training program to a level associated with 
not being overrepresented in the lowest one-fourth of job performers. 

It should be noted that, in the foregoing case, the puAp06Z of the 
exercise in estimating the reading demands of jobs — in this case to 
derive a goal for reading training — entered into the formulation of the 
decision rules for setting criterion levels of job proficiency and for re- 
lating reading skills to those criteria on both job sample and job knowl- 
edge tests. In the project under discussion, all persons tested were, in 
fact, working in their jobs and, by virtue of this fact, could be construed 
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as successful job performers. Hence, the decision rules had to be formu- 
lated to discriminate among Successful job performers for the purpose of 
deriving goals for reading training. If the purpose of the determination 
of reading skills demanded of a job is to screen out people or classify 
them for limited assignments, then a different criterion (e.g., attrition) 
and different decision rules would be called for. 

While the method of correlating reading and job proficiency criteria 
IS a general approach with established methodological techniques, it has 
some serious limitations. For instance, because the job proficiency meas- 
ures are likely to be only indirectly mediated by knowledge which may have 
been previously learned by reading, relationships of reading to job sample 
performance should be expected to be smaller than relationships among gen- 
eral reading measures and job reading tasks or other paper-and-pencil 
measures of job proficiency. These considerations, plus the fact that the 
costs of constructing and administering an extensive job-sample test to 
joD incumbents in a representative sample of jobs are prohibitively high, 
would seem to mitigate against the use of job sample criterion tests for 
all but fundamental research purposes. 

Correlating reading skill with job knowlzdqz is a standard, straight- 
forward approach to determining job reading requirements which might readi- 
ly be used in various job or training settings with existing personnel job 
Knowledge tests. However, it may not be certain that reading test scores 
are correlating with job knowledge per sa - they may be correlating with 
tae reading demands 'of the job knowledge test. Application of this method 
on a broad scale would seem to be justified only if it could be shown that 
job knowledge, and not general reading skill per se, was needed for scor- 
ing well on the job knowledge tests. 

_ Furthermore, the method of correlating reading skills with job pro- 
ficiency measures such as job sample tests, job knowledge tests, attrition 
and similar criteria to establish reading levels of jobs provides no direct 
indication of how well a person must read to perform job fiZaduJlQ tasks. 
Many job tasks can be performed without reading, and may be learned by 
watching and imitating others. For this reason, reading demands of jobs 
may be oyer- or under-estimated whenever non-reading criteria are used to 
establish relationships of reading skills to job proficiency. 

Estimates of Reading Demands of Jobs Derived from Job Reading Task 
rests_: To render the correlational analysis technique more directly rele- 
vant to job KZad^ng demands, it may be possible to relate general reading 
skill to performance not of job tasks in general, but rather to perform- 
ance o. job KdddaiQ tasks; i.e., tasks in which reading is required. 

_ In research of this nature for the Army (Sticht, 1975c) Cooks, Mech- 
anics, and Supply Clerks were interviewed at their job sites. In the 
interview, each man was asked to identify reading m.aterials he had used 
an performing some job task. Copies of these materials were obtained and 
analyzed as to the reading tasks involved in using them. Tasks such as 
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reading^^ tables of contents", "reading indexes", "reading procedural di- 
rections , reading tables of standards and specifications" were identi- 
xied. Special job reading task tests (JRTT) were constructed which tested 
-lie ability of men to perform the different job reading tasks. 

r^ A A^"^ ^^^^ ^""^ ^ general reading test were administered to several hun- 
triw^ personnel. With these two sets of scores, it was then possible 
relate various criterion levels of achievement on the JRTT to the gen- 
eral reading grade level o.: ability needed to achieve this criterion. 

Using this approach, and by averaging over the various reading tasks, 
possible to indicate general reading levels associated with various 
c.-terion levels of performance on the job reading tasks as a group. 
^ivt.n this information, and a decision about the criterion level of per- 
formance which job performers or job aspirants should display on the job 
reading tasks, a general literacy requirement can be estimated for each 

JOD. 

In the Army research, it was found that if a criterion of excellence 
^-as cnosen such that 70% of the people were expected to get 70% correct 
on the job reading tests, the general reading requirements for the Cooks 

cte;t''^%9 "^'i'"^? ^^y^^' J^^^hanics - 8th-grade level, and for Supply 

L*u.erKs - li~grade level. 

^ The job fizadlHQ ia^k tut rrntlvr represents the most direct approach 
to aeterminmg job reading requireir.ei. in that it takes as its criterion 
r.ea:^-.-.-. xhc reading score on the JRTr, a sample of actual and commonly 
u-.ed o:. reading materials. To the extent that the job reading passages 
constituting the JRTT .omprise or represent all the reading tasks of th^ 

TPTT ^''f^'"' ^^""^ ^^^'"^ P^°Pl^ ^^^^d to perform on the 

^fP^^^^^t t^sks people have to perform on the job with those materi- 
al., then the ability to read the JRTT passages is the ability t- perform 
^hc job reading tasks and thus, to meet the job reading requiremei'.t . 

, .,/f'^^^^°"^-ly' it should be noted that using the JRTT method, readirc. 
skill level requirements will change depending upon the criterion of per- 
fonr.ance_ selected. This latter problem, i.e., the problem of specifying 
a criterion, must be dealt with in any approach'to the determination of 
reading requirements. in which criterion performance measures are obtained 
ihe question is one of "how good is good enough"? It is possible to say 
u.xat all people should be able to perform all reading tasks with 100% 
mastery. But If there are restricted manpower pools, and if many job 
reading tasks are quite complex, this would seem an unrealistic goal. 

The Valid ity of Estimates of Reading Demands of Jobs 

From the foregoing discussion it should be apparent that there is no 
such empirical "thing" or "stuff" or "event" or "condition" known as 

■cnz reading demands of a job". Reading demands are not cLL6C0V2Ae.d, they 
arc cA.£a^£d by procedures which are more or less systematic and performed 
according to more or less specifiable rules. The question of the vaLicLUy 
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of £iny estimate can only be answered with respect to a model or theory 
of job-related reading which "v/ould define systematic procedures for ob- 
taining estimates of reading demands of jobs for various purposes permit- 
ted by the theoretical constructs involved. 

In the absence of such a theory, it may be easier to know that an 
approach is not valid than to know that one is. For instemce, the ques- 
tion of validity seems clear with the readability approach: it in no way 
involves figures and tables in the estimate; and Sticht, et al, (1976) 
found iti their stnactured interviews with Navy personnel that only 32% 
of the reported reading tasks involved texts only; 62% used figures or 
figures and texts combined (the remaining 6% was "no response"). 

The job reading task test method demands a theory for constructing 
reading tests which can be applied to a sample of reading tasks. But, 
if we are to be certain that these reading tests match the information 
processing which was done in the actual performance of the reading task 
on the job, then the theoretical constructs in the "reading test model" 
must be incorporated into the interviews or other approaches used to ob- 
tain the sample of reading tasks , because , since reading is a "private" 
act, it is necessary to query people about their information processing 
during the performance of a reading task in order to discover whether or 
not they perform the types of information processing involved in the mod- 
el of reading. 

Perhaps the most valid estimate of the reading difficulty level of 
military jobs would be obtained by having personnel of various reading 
levels perform the actual job tasks and reading sub tasks which were re- 
ported to have been performed. But, what would be scored? Completion 
of the job task? Perhaps some could be completed even though some or all 
of the reading mater:^.al was not understood. How would one know? Would 
speed of reading be scored? Would it matter? Could ^ne then, develop an 
inventory, as in the Canadian study, for general use in determining read- 
ing demands of all jobs which job analysts could use in a discriminating, 
reliable manner? How would this be determined? 

These considerations indicate some, though clearly not all, of the 
conceptual and methodological problems involved in the use of the strat- 
egies of nonaccep tance and limited assignments based on reading require- 
ments of jobs for coping v/ith problems engendered by low literacy skills 
of personnel. Hopefully, awareness of the procedures and problems dis- 
cussied here will be useful in future research to develop more adequate 
techniques for the assessment of the reading demands of military jobs. 
Additional discussion of these types of problems can be found in Bormuth 
(1975). 
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Modification of Training and Job Requirements 

A third strategy for dealing with the problems of low literacy skills 
is to redesign training and job materials to minimize the importance of 
such skills. Urider this strategy, training schedules and practices may be 
iTiodified to meet the skill levels of different individual--. Thus, written 
iastructional material may prove adequate for certain individuals, but the 
sac^e instructions might best be presented in some other way for individu- 
als having relatively low reading skill levels • 

The application of this approach is illustrated by consideration 
of two primary recommendations regarding the training of Mental Category 
IV personnel presented in a review of some five years of study by the 
Naval Personnel and Training Research Laboratory. 

1- Do not assign Group IV personnel to Navy ratings that 

have high verbal, computational, or conceptual require- 
ments. (The storekeeper rating is an example.) 

2. In general, training programs should minimize reading 
requirements (including written tests) and theoretical 
knowledge and should utilize pictorial materials, simpli- 
fied terminology, and practical illustrations. The 
actual performance of job skills should be stressed. 

(Steineman, Hooprich, Archibald, 5t Van^Matre, 1971, p. 8.) 

The Na-^/y laboratories are not alone in their conclusions regarding 
the advisability of usin.; reading materials for training Category IVs. 
In a report of an extensive series of studies on the development of train- 
ing techniques for use with Category IV personnel. Army research concluded: 

For tha low aptitude groups, printed programs or programmed 
texts were the least effective technique. On the basis of this 
study we conclude that written material as the primary medium 
should be used with low Category IV personnel only as a final 
recourse. (Bialek, Taylor, St Hauke, 1973, p. vii.) 

Papers in the present volume present evidence of the concern for 
reading demands of jobs and job training programs by reducing the reading 
difficulty level of materials. (See papers by Burkett, Curran, Hiller, 
Johnson, Kern, and Muller.) As these papers suggest, lowering the diffi- 
culty level of the printed materials is an important step. However, *-his 
approach, by itself, cannot be expected to ''solve'' the literacy problem. 
The extent to which the reading difficulty level of technical materials 
can be lowered is limited. After a certain point, it becomes necessary 
to start simplifying and deleting^the more demanding job knowledge re- 
quirements. The result can be to render a person useable only in a very 
limited job assignment. This may be a satisfactory state of affairs when 
manpower is unlimited and jobs can be fractionated and specialists who do 
nothing but a small part of the job can be used. However, we re in a 
time in Xi/hich weapons systems and associated publications systciins are 
growing more complex and voluminous, as Mailer's paper in this volume re- 
.ports. Accompan^^jng this growth in weapons/docim^.ents systems has been a 
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reduction in the numbers of personnel in the Armed Services. The fact 
that there is more to do with -fewer people precludes the "divide and con- 
quer'' strategy involving more and more specialists. In fact, what is 
needed are mere and more generalists, and to have flexibility, a person 
must have the fundamental literacy/cognitive skills to adapt to new job 
demands as they occur. 

On the other hand, as Glaser comments in this report, caution is 
required to assure that job training prograais are not overly complicated 
by the excessive use of language — written or spoken* Olson's (1975) 
trenchant discussion of the problem of over-emphasizing literacy skills 
in training programs is particularly apt. 

The nature of schooling per se tends to make literacy 
relevant to man}?" nonliterate tasks simply because the means 
of instruction, worked out to permit the group instruction of 
large numbers of students, tends to require literacy. Many 
tasks, if taught in a practical context, on a master-apprentice 
basis, allowing for attempts at performance accompanied by feed- 
back, could be taught with very low literac}' demands. To teach 
the same tasks on the basis of group instruction immediately 
transforms the task into a liberate one — presumably this is 
thfe case whether' or not one uses printed materials or lecture 
materials. In both cases, a practical task is changed primarily 
into a verbal one — calling for the analysis and memory of 
.-statements instead of a pattern of action. This is well illus- 
trated in the familiar poem by Henry Reed, called Naming of Parts. 
The poem illustrates our problem. The use of a rifle by a 
soldier .is certainly a practical action. But to permit group 
instruction in its use, and perhaps to permit group performances 

certain kinds, the practical action is dissolved into dozens 
of verbal descriptions which are then memorized. In this way, 
practical action is transformed into a task of verbal learning. 
Once any practical action is so transformed, literacy skills 
become relevant to the acquisition of practical action. Hence, 
the reportedly high correlation between literacy skills (IQ 
roughly) and success in such things as military training. But 
that correlation occurs, it may be argued, simply because of 
the form of instruction employed; it may not occur with the 
practical actions themselves. . . . Here is Reed's poem: 

Today we have naming of parts. Yesterday, 
We had daily cleaning. And tomorrow 
morning 

We shall have what to do after firing. But 
today. 

Today we have naming of parts. Japonica 
Glistens like coral in all of the neighboring 
gardens. 

And today we have naming of parts. 
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This is the lower siding swivel. And this 
Is the upper sling swivel, whose use you will 
see 

Wiien you are given your slings. And this is 

the piling swivel 
Wiich in your case you have not got. The 

branches 

Hold in the gardens their shy, eloquent | 
gestures , 1 
Which in our case we have not got. 

This is the safety-catch, which is a:lways 
released 

With an easy flick of the thumb. And please 

do not let me ' j 

See anyone using his finger. You can do it ! 

quite easy j 
If you have any strength in your thumb. 

The blossoms • 
Are fragile and motionless, never letting i 

anyone see . ; 

Any of the^Ti using their finger, 

t 

And this you can see is the bolt. The purpose ; 
of this 

Is to open the breech, as you see. We can j 
slide it 1 

Rapidly backx^ards and forwards; we call this ; 

Easing the spring. And rapidly backwards and 
forwards 

The early bees are assaulting and fumbling 
the flowers: 

They call it easing the Spring, i 

They call it easing the spring: it is j 

perfectly easy ^ \ 

If you have any strength in your thumb: • like ! 

the bolt, I 

And the breach, and the cocking-piece, and 1 

the point of balance, ' 

Which in our case we have not got; and the ! 

\ almond-blossom 1 

Silent in all of the gardens and the bees i 

going backwards and forwards, ! 
For today we have naming of parts, Henry Reed 

There are two points to be made. First, the very demands 
of large group instruction tend to translate even practical 
tasks into highly literate ones- Such translation is prejudi- 
cial to people with lower degrees of literacy — such learners 
will make slow progress and will be regarded as dull, even if 
they ultimately make" the best performers, (Olson, 1975, pp. 149-152 
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VPnllQ there can be no denying that marginally literate personnel can 
be traiaed in motor skills using "hands-on'' methods and utilized in less 
demanding occupations, there is reason to question the long-teriJi value of 
the strategy of nonverbal training, both from the Services' perspective 
and that of the trainees. For one thing, though reading materials can be 
eliminated from training programs, it is doubtful that they can be removed 
from on-the-job use. The reason is that printed materials serve as repos- 
:itoriGs of needed facts (e.g., standards and specifications, procedural 
directions) which, if not accessible by means of reading skills, must be • 
nisr.orised in training. Since equipment and procedures may change Ci{^teA 
a man has completed training, it would prove very expensive to redesign 
a training program to provide nonreading , hands-on training every time 
such changes occur. A primary function of print is, in fact,, to replace 
the need for direct, hands-on, personalized instruction, thus expanding 
the individual's rajige of utili^-^ation. 

With regard to career development, the strategy of avoiding the use 
of printed materials, starting early in a man's career, may foretell a 
limited level of achievement later on when* actual job demands call for 
greater use of reading materials, as indicated above. 

The problem for the design of training is to insure that the train- 
ing prepares personnel for performing all entry-level tasks - including 
both motor skill and literacy tasks. 

Remedial Training 

A fourth strategy sometimes used by the Services in an attempt to 
solve the problems associated with low literacy is to provide remedial 
training in reading. The chapters by Fletcher, Hiller, and Burkett review 
remedial reading training efforts in the Services up to the end of the '50s 
while Caylor, Duffy, and Groff discuss current literacy training programs 
in the Army, Na^/y, and Air Force. 

For the most part, literacy training in the Armed Services has been 
closely associated with major mobilization efforts; namely. World War II, 
the Korean Conflict, and Vietnam. During sucH efforts, large numbers of 
personnel are needed, and it becomes necessary to recruit large numbers 
of persons having low literacy skills. The typical response, then, has 
been to initiate a remedial reading program to train these people to read 
at some minimal level p/iCAumQ.d adequate for coping with the basic demands 
for reading in the military. For the basic reading programs established 
during Project 100,000, the targeted levels for reading achievement ranged 
from grade 5.0 to 6.0 across the three Services. 

Only a limited amount of data bearing on the effects of military lit- 
eracy training programs on job proficiency have been found. Much of the 
research to evaluate literacy training conducted prior to Vietnam, sum- 
marized in detail in the book McUiglnaZ Man and hiiLUoAy SqAvIcq. (Depart- 
ment of the Army, 1965), failed to demonstrate benefits of literacy train- 
ing on either job training or job performance, measured in various ways — 
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supervisor's ratings, retention, pay scale achievement, conduct, and 
others. However, these studies also failed to demonstrate that the vari- 
ous literacy training efforts improved reading skills over those of con- 
trol groups — where such comparisons were made. If proficiency in 
reading is not genuinely improved, it is meaningless to search for the 
ej.iects of such non-improvement on subsequent measures of job training 
or" job perfomance proficiency. 

_ Evaluations of literacy programs since Vietnam with respect to their 
mpact on training or job proficiency are scarce, and in no case do. they 
isolate the effects of literacy training per se from the effects of having 
an additional 4 to 8 weeks of time to adjust to military life and, in most 
cases, to receive additional training in military subject matter offered 
during the period of literacy training. 

In the case of the Army Preparatory Program initiated at the outset 
of Project 100,000, Fisher (1971) compared men who were successful and 
unsuccessful in reading the fifth-grade level of reading and found that 
they did not differ greatly on most indices of military status and per- 
formance, though successful trainees were slightly more likely to achieve 
a higher pay grade and to be judged eligible for reenlis traent. 

In an unpublished report by Zaccaria (1971), graduates of the Air 
Force remedial reading program showed a much larger completion rate for 
basic military training than comparable (in reading) personnel who did 
not have remedial reading training. However, the remedial reading program 
aliio offered four hours of military training daily, so it is not certain 
that the reading training per se resulted in the different success rates 
in basic military training. 

Duffy's paper in the present volume points out the inconclusive 
nature of data on the impact of the Marine Corps and Navy reading pro- 
grams on later Navy effectiveness. Apparently, adequate studies of this 
relationship are not available. 

Though it may seem a fairly straightforward problem, the evaluation 
of the effects of literacy training on job training and job performance 
proficiency is fraught x^^ith many difficulties. X-Jhat should the measure(s) 
of job proficiency be? If hands-on job sample tests are used, is there 
any reason to believe that limited literacy training will have any effect 
on such performance measures? Work by the Human Resources Research Organ- 
isation (Vineberg & Taylor, 1972) compared the job proficiency of men 
given job training (8 or so weeks of school training in the performance 
of job tasks, not reading training) as Cooks, Vehicle Repairmen, Supply- 
men,, and Armor Cre^^men to' that' of men given on-the-job training. The pro- 
ficiency measures were job sample tests based upon task analyses of these 
jobs. Results indicated that, for Repairmen and Cooks there was a statis- 
tical (but not practical) difference in favor of the men given formal job 
training, while there was no statistical or practical differences between 
•the formally trained and on-the-job trained Armor Crewmen and Supplymen. 
(In these comparisons, effects of AFQT, time on the job, education, and 
age were controlled by covariance techniques.) 
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If eight or more weeks of formal job t/iCullvinQ may have little or no 
effect on. subsequent job sample test performance in these types of jobs, 
is there any reason to believe that four, six, or even 13 weeks of /izadLing 
training will be reflected in such measures? Probably not. Many job tasks 
can and will be learned by "show-and-tell" — especially tasks of high 
frequency ^nd criticality ~ i.e., the types of tasks emphasized in task 
analyses and typically used to construct job sample tests. 

Furthermore, except for the p^^o^oundty illiterate (e.g., persons 
reading belcw the grade 2.0 level) which most of the military reading 
students are not, it seems unreasonable to expect four to six weeks of 
literacy training to meaningfully affect retention rates, rate of promo- 
tion, conduct (AWOLS, court-martials), paper-and-pencil job Icnowledge pro- 
ficiency test scores, or supervisors' effectiveness ratings. For the 
pW^OuyidZy ^UZLteAaXz, improvement on any of these indices would be con- 
tingent upon successful literacy training up to some requisite level. 
The Air Force report by Zaccaria (1971) suggests that it is difficult to 
effect much reading improvement with such personnel in as much as 13 weeks 
of reading training. In this study, 85% of the men who entered reading 
at the first-grade level exited reading at or below the fourth-grade levels 

Perhaps the most reasonable criterion measure pf job proficiency 
against which the effects of reading training might be evaluated would be 
measures of a person's ability to perform necessary job tasks involving 
reading. H(ic.U6cUiy reading tasks could be defined as those reading tasks 
which a person would have to perform he was working alone on the job 
and had no one to turn to for help in doing a job task. This is a- some- 
what artificial situation, since it is rare for personnel to work under 
such restrictions. Usually, the marginally literate person can rely upon 
someone more literate to handle the reading needed in various job duties. 
But if it is desired that each individual be able to perform aZt signifi- 
cant job tasks, including the reading tasks, then it seems reasonable to 
evaluate the effectiveness of literacy - training by comparing the perform- 
ance on job reading task tests of graduates of the literacy training 
program with that of persons of equal pre-literacy training reading skill 
levels. It seems reasonable to argue that if reading training does not 
improve a person's ability to perform significant job reading tasks, then 
it is unlikely that the reading training will affect other, less directly 
reading-related indices of job proficiency, such as retention rates, 
supervisors* evaluations, job sample tests, etc. On the other hand, it 
does not necessarily follow that improved competency in performing job 
reading tasks will be reflected in other, non-reading related indices of 
job proficiency. In fact, data from KumRRO (Vineberg, Sticht, Taylor, & 
Caylor, 1971) indicated that job knowledge and job sample test performance 
for the four jobs mentioned earlier correlated only about .5, while super- 
visors' ratings were only trivially correlated with either of the other 
indices of job proficiency. There is thus evidence to suggest that im- 
provement on one index of job proficiency - e.g., the use of job reading 
mate.rials — will not necessarily be reflected in improvements on other 
indices of job proficiency. 

33 



34 



To date then, tha strategy of dealing with literacy problems in the 
Armed Services by providing brief remedial reading programs has not been 
demonstrated to be an effective approach, to developing more competent, 
effective readers capable of contending with the readirg demands of tech- 
nical training programs and many reading tasks demanded by military jobs. 
Usually, only one or two years of gain in "general" reading proficiency 
ara produced, which may typically shrink to less than one year after a 
few weeks following the reading training (see the papers by Duffy and 
Caylor). 

It seems likely that the belief that adults can be taught to read 
in brief periods of time is due to our nut having adequately understood 
what ve mean by the concept of "literacy", A major (though not the only) 
problem is the tendency to confuse the information processing skills of 
reading and xvrriting with the- knowledge expressed or received by those 
skills*. For example, in a reading vocabulary test, it is possible to 
miss one item because one lacks knowledge of the word meaning. A person 
might be perfectly capable of saying the word out loud, however, thereby 
indicating skill in converting the. printed word into a spoken word. 

It is clear that if a person does not know the meaning of a v7ord, 
then our instructional practice must differ from what we would do if the 
person knew the meaning of a word when that word was spoken, but could 
not recognize the word when presented in the printed form. If a person 
does not know the meaning of a v7ord when presented in the printed form, 
and also does not know the meaning, of a word when presented in the spo- 
ken form, then we may conclude that the person will have difficulty un- 
derstanding reading material containing that word, but the problem will 
not be a reading problem. Rather, it is a language problem which no 
amount of instruction in sight/sound correspondences or sight/word rec- 
ognition xvn,il remedy. 

This distinction between the oral language knowledge a person has and 
skill in recognizing the same language form in print is a crucial one for 
understanding why brief reading programs are likely to produce only slight 
improvements in the ability to perform various, tasks involvJ.ag reading. 
The unspoken assumption of reading training programs for adults is that 
many illiterate or marginally literate people have a fairly well devel- 
oped oral language capability. Hence, a relatively brief period of train- 
ing in decoding skills, i.e., providing knowledge of sight/sound corres- 
pondences and opportunities for practicing this knowledge to develop skill, 
is considered sufficient for unlock^.ng the person's oral- language skill 
via the printed word. At the extrertna, we might imagine a person whose 
oral language skills /knowledge were equivalent to those of a typical 18 
year old with 12 years of education, but who had simply neglected to learn 
to read. -In this case, a relatively brief period spent learning the heur- 
istics of decoding would permit this person to bring this fully developed 
language skill to bear on the printed word. In such a case, .the person 
might gain as much as 12 years in reading skill with only a few weeks of 
decoding training! 
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A problem for adult literacy programs is that clients are likely to 
be poorly developed in botli-oral and written language skills. In Air T'orce 
sponsored research (Sticht & Beck, 1976) men in a military literacy train^ 
ing program were administered a test of auding-^and reading comprehension, 
iae results showed that both oral and x/ritten language skills were at the 
^th grade level. These findings are confirmed in research reported here-- 
m by Duffy using Navy personnel. 

The fact that most marginally literate personnel score several years 
below^^the norm for their age group in auding' indicates that their "lit- 
eracy^ problem is not restricted to the printed page. Rather, it reflects 
in la^ge measure a language processing/comprehension problem. Low oral 
vocabulary scores indicate a lack of language knowledge, not just low 
skill in processing the connected discourse used in assessing paragraph 
couiprehension. Because language knowledge is low, reading comprehension 

^° Hence, eve^n if a marginally literate person could 

decode printed language as well as he could spoken language, his low oral 
language skill would retard comprehension. An implication of this is that 
major improvements in reading skills of marginally literate personnel will 
require major improvements jun language competence (e.g., vocabulary 
knowledge) . 

^ Carroll (1971) discusses problems involved in measuring growth in 
vocabulary (by auding or reading). He cites work by Edgar Dale in which 
^s^i^^tes that children may finish first grade with a vocabulary of 
-3,000 V7ords. Then he estimates that they will add about 1,000 words per 
year from then on through high school, with high school seniors knowin^^ 
about 15,0C0 words. 

Accepting this rate cf growth, if marginally literate personnel typ- 
^■^nnn^ ^'^'^^ vocabularies like beginning 5th graders, then they know about 
0,000 words. To achieve a 6th-grade vocabulary, about 1,000 x^7ords will 
have to be learned. In a 6-week program in which six hours of active 
learning occurs each day (an unlikely situation) new words would have to 
be learned ayid /i^;tcUnQ.d at a rate of six per hour. To reach a 7th-grade 
level vocabulary, 12 words per hour would have to be learned and retained. 
The problem enlarges when it is recognized th^t this rate of learning is 
based upon typical learning rates of typical children. Marginally liter- 
ate personnel, however, usually require anyxjhere from two to four times 
as much time to learn verbal concepts as typical recruits do (Fox, Taylor, 
& Caylor, 1969). For these people,- it seems unlikely that they could 
learn and retain six to 12 v/ords per hour for six hours a day, or even 
half this number. As Carrol has pointed out '\ . . basic linguistic com- 
petence (at least with respect to grammar and vocabulary) is probably 
relatively unsusceptible to improvement except over long periods of time 
and with tremendous efforts . . . (p, 130) . 



1^ ^. . 

Kuaing rerers to listening to speech in order to comprehend and is a 
parallel term to reading, which refers to looking at printed language 
in order to comprehend. 
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The data regarding the effectiveness of current literacy training 
programs reported in Part IL of this volume, and the foregoing consider^ 
atxons strangely suggest that the brief literacy programs the Services 
have operated in the past and are currently operating take only the first 
few steps of a long journey. Furthermore, it does not seem unreasonable 
to expect that evea the progress gained in the current programs will be 
lost because no systematic efforts' are made to continue to stimulate the 
less literate man to continue his reading development after completion 
or literacy training; no job-^related , developmental reading materials ar^ 
provided, and no o '^icially scheduled time for continued literacy devel- 
opment is provided. 

A similar concern for continued education and training in reading 
following the initial short-term program has been expressed by Kent et al 
{IJ/l) as a result of their examination of civilian job-related adult 
basic education (ABE) programs. 

Available figures indicate that a single, ABE program by 
itself usually accomplishes very little. In most programs, 
the advances which occur in reading, ^^riting, and arithmetic 
are somewhere around one grade level, or occasionally two 
grades. Advances in self-confidence may or may not be more 
substantial — no figures are available. But new self- 
confidence, even if present, ic> probably fragile. 

So the ABE graduate is unlikely to be in a markedly dif- 
ferent employability situation than faced him before ABE. 
(^^ote the significance for ABE program goal definition — that 
program goal statements should refer to making a small start 
in an important direction, and that programs should be evalu- 
ated not so much by the amount of the start, as by the real- 
istically useable momentum imparted in the right direction and 
by their working linkages with post-ABE activities.) Post-ABE 
help is required to capitalize on these small but important 
gains. Either the ABE program should have ski3.1s training and 
job placement or advancement built into it, or (more to the 
point) ABE should be built into a total emplojrment or advance- 
ment program. If the ABE is realistically related to what is 
going to happen next in work or training, then its results 
will be reinforced and expanded upon naturally by the new 
work or training situation. 

Evaluation of a well organized total job-related ABE 
program will result in recommendations for improvements, 
perhaps to include additional time with ABE content at later 
dates (say 3 months, 6 months, or a year after "graduation"). 
These later sessions might be even more thoroughly integrated 
with job training than the original sessions were. 
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Here we are pointing out the importance of a context for 
ABE which will link it xdlth skills training and work. This 
context can hardly be supplied by ABE itself, but it could be 
supplied by skills training or by work or by some combination. 
Many unions and employers have ongoing training programs, in- 
cluding classroom work and OJT, to which ABE could reasonably 
be adapted and which could supply this ongoing post-ABE support. 
Also, many schools (adult vocational schools, community colleges, 
and technical institutes) could (if closely enough related to 
employment situations) provide support. Any of these types of 
post-ABE support could also be made a part of the "selling- 
package" for ABE to increase its motivational attractiveness 
to students. 

ABE is only a beginning, and much more beyond ABE is 
needed. But for many ABE students, a long-range view is 
entirely uncustomary. The ABE program should be set up so 
that it can lead on and on, but it may be inadvisable to 
burden the student with too much thinking about the future. 
For motivation and symbolically, glimpses of a distant future 
may help. But detailed learning activities should usually be 
based on more immediate student interests." 

(Kent, Bishop, Byrnes, Frankel, 
Herzog, & Griffith, 1971, pp 94-95.) 

RESEARCH TOWARD A COMPREHENSIVE CAREER-ORIENTED 
LITERACY TRAINING SYSTEM FOR THE ARMED SERVICES 

Because of the many conceptual and methodological problems mentioned 
above, and the increasing recognition that the current, brief, one-shot 
remedial literacy training programs are inadequate for the task at hand, 
the Air Force , Army , and Navy research papers reported herein by Groff,' 
Caylor, and Duffy each call for research which would study methods for 
validly characterizing literacy demands of jobs, and for the design and 
development of literacy training programs which systematically teach ^ob- 
related vocabulary and reading tasks, and which are extensive enough to 
permit a long-term period for the development of literacy skills. 

The interest in teaching job-related vocabulary and reading tasks 
stems from the desire to have personnel rapidly acquire the language 
needed for their military work. Since "general literacy" is made up of 
specific vocabulary, the teaching of job-specific vocabulary adds to a 
person •s^^"general literacy" competence. However, the teaching of "general 
literacy" vocabulary, in a non-job-focused manner, will not necessarily 
induce the various technical words and concepts found in military train- 
ing programs and jobs. For these reasons, there seems to be a general 
consensus among military educational researchers that military literacy 
training ought to be focused on job-related reading requirements (see the 
papers by Caylor ,^ Duffy , and Groff). This will both improve a person's 

general literacy" ability, and permit the more direct transfer of skills 
from the literacy program to the job situation. 
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Recently completed research, by Sticht et al. (1976) for the Navy sug- 
gests the possibility that basic military trainijig, job technical school 
training, and correspondence course materials may be modifiable such that 
literacy sKills and job skills training can be offered in an integrated 
opf'r^'^J^" concept is based on findings which suggest that much, perhans 
^3/o uo .-50/0 of wnat is currently taught in these training programs, is not 
considered relevant to their work by Navy personnel. That is, much of the 
current training programs may consist of "nice-to-know" but not "need-to- 
know information. Thus, some instruction time might be saved if the 
training programs were systematically redesigned to effect a better match 
between training and job requirements. 

Additional time savings might be accomplished by converting tra-fning 
programs from lock-step, lecture-oriented classrooms, to individualized, 
self-paced programs ;7ith flexible entry and exit. Where this has been 
done elsewhere in military training programs, considerable overall sav- 
ings in man-hourj (and hence cost) of training have been affected. 

Because of the time, materials, and cost savings which may result 
-rom the careful systems engineering of training, and the move to self- 
pacing or instruction, it should be possible to incorporate literacy 
training into the programs for those who need it. \^Sle they woul-^ 
necessarily require more time to complete training urograms under this 
system, the expectation is that overall tra-ining costs wil]. drop, as 
unose capable of accelerated programs are permitted to move rapidly 
tnrough the streamlined programs. 

Figure 1 presents a .general plan for research and development to'' 
produce a comprehensive, integrated job skills and literacv skills car- 
eer development system. The first phase of the plan calls' for a thorough 
analysis of reading within the Services, including the determination of 
genera, reading levels for performing job-related reading tasks so that 
reading training objectives may be targeted to job requirements. 

An additional aspect of the Phase 1.0 activities given in Figure 1 
IS tne study of job skills training and the general education system with- 
in the Services to determine the feasibility of improving linkages in these 
systems, and potential for revising these systems to produce a more inte- 
grated job skills and literacy skills training system. The report by 
bticht et al. (1976) illustrates the type of approach envisaged under the 
Pnase 1.2 activities and provides information concerning the Navy's cur- 
rent career development system, including information regarding job skills 
training, career counseling, and general educational systems. A general 
rmding is that the linkages among these^ subsystems are very loose - if 
not tonally absent. Initial efforts toward a comprehensive career devel- 
opment system which includes integrated job skills and reading skills 
training should include the formation of tighter administrative and con- 
tentive linkages among these subsystems. 
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PHASE l.C 



1.1 DETERMINE READING DEMANDS 1.2 STUDY JOB SKILLS TRAINING 
OF MILITARY JOBS | & GENERAL EDUCATION ■ 

SYSTEMS 



PHASE 2.0 



2.0 DESIGN & DEVELOP INTEGRATED JOB SKILLS & READING SKILLS 
TRAINING SYSTEM 



2.1 INTEGRATED BASIC MILITARY TNG/READING TRAINING 



2.2 INTEGRATED JOB SKILLS/READING TRAINING 



2.3 INTEGRATED ON-THE-JOB TPJ\INING/READING TRAINING 



Figure 1 . General Plan For Developing An Integrated 

Job Skills & Reading Skills Training System. 
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The Phase 2.0 activitiea of Figure 1 call for the evolutionary devel 
opment of integrated job skills and literacy skills training programs for 
the three major stages of military cape^ei development: entry into the 
Service with basic military training; job technical school training; and 
assignr.ent to on-the-job training s.t an active duty station. By provid- 
ing opportunities for literacy training at each of the major career devel 
opment stages, ample time is available for accomplishing genuine improve- 
ments in literacy skills. 
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RESEARCH AND THE TRANSFORMER: A PROGRAM FOR 
'IMPROVING TEXTS 

Michael Macdonal.d-Ross 
The Open University, Institute of Educational Technology 

In a inodern industrial society such as. ours people often get the 
information they need from dociimentary sources — pieces of paper or 
computer files. People Ccinnot always meet face to face, and there can 
be quite a distance between an expert source and the user or client. 
It makes sense for us to spend time and effort on the business of com*- / 
Tnunication. The task we face is simply this: koiJO cum We o/Lga^U-6e com- 

VHien we see one person talking to another, both participants employ 
complex techniques so as. to make themselves understood. These techniqi/es ' 
of interaction may have taken millions of years to evolve, but are de- 
ployed instantly without self-conscious thought. It is. much more diffi- 
cult to make oneself clear to a reader who cannot see and who cannot ask 
questions. So, as one might expect, domnuyiiccution b/ikcdzdo'JyUi are quite 
cornmon, and can have the most serious effect.. In England we have been 
suffering from what is known as Dutch Elm Disease — a, fungal infection 
carried by small beetles. To stop the disease spreading, farmers were 
iidvlSL u to cut down any infected trees. But the Government circular for- 
got to tel.' them to bum the wood! I do not know what the candidates 
might be £ar the most costly communication breakdown in the history of 
the human race it is interesting to speculate — but I can tell you 

that our little beetles are still spreading merrily through England de- 
stroying elm trees as they go. And, how often have we met forms we could 
not fill in or regulations we could not properly unde?:stand? How often 
are service manuals unuseable? How often is. our printed communication 
useless except to those who already know the answers? This sort of thing 
occurs too often and is too serious for us to accept complacently. We 
must do something about it. 

For the past year my Research Assistant, Robert -.Waller , and I have 
spent many hundreds of hours investigating the ways in which complex 
printed ' communication could be improved. (We have reported some of the 
preliminary results of these enquiries in Macdonald-Ross , 1975; Macdonald- 
RosG & Waller, 1975a, b, 1976.) The work'falls into two parts.: that 
v/hich concerns "the ' research v/orker, and thit which concerns the practical 
communicator the editor, designer, author ,j-;/or whatever. 
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The trouble with research is^. . . 

Researchers often talk as though they had a great deal of knowledge 
which was goiiig to waste; they suggest that practical people outside uni- 
versities are stubborn or, pigheaded because they go their own sweet way 
and take no notice of what the researcher says. Now, it is very conven- 
ic^nt for the researcher to put the word about, that research is very sue-- 
cessf^jl and is leading to deeper understanding and, a changed world/ But, 

is the actual reality? Just how much is known and what may we ex- 
p^o- of researchers? Let us look into the matter together with the help 
of some examples from the field of graphic communication.* 

I'll start with the research into typography. This is a well-estab- 
lished and apparently well-researched area; the key reviews are Pyke (1926), 
Paterson and Tinker (1940), Luckiesh and Moss (1942), Burt (1959), and 
Tinker (1953). We reviewed this work i\i detail and- began co identify sys- 
CGu^atic defects v/hich we documented aua attempted to explain. In the first 
place, there are many experiments ^^hose relevance is quite obscure. For 
er.ample, there are many tests for the relative * legibility of characters 
in a fount. Lists of "letters of low legibility" or "pairs of letters 
most, likely Lo be confused" abound in tihe literature. But, no-one tells 
UG what we should do with such results arid i suspect this is proper caution, 
tor xt has long been kno^vm that occasional confusion of individual j.etters 
xr. nor of prime importance in the reading process.' 

Much more im.portaht are the omissions. There is a sense in which 
legibility^. rresearch has hardly started — so many important topics have 
been tackled. There are different kinds of omission. Sometimes previous 
research needs to be updated; for example, tests done on obsolete type- 
faces need to he redone on new typefaces. ''But a more important omission 
by far occurs when researchers overlook the advent of a completely new 
problem. For example, program learning brought with it questions and an- 
swers (prompts and responses) ; yet there is no work on how to set such 
items in relation to the rest of the text. Modem science textbooks rely 
on a close interdependence .of text and diagram; yet, again, there is no 
research to guide the designer or the typographer. Errors of irrelevance 
and errors of omission are, of course, connected. The same set that pre- 
disposes researchers to choose problems that can he neatly handled in a 
laboratory, also prevents them ,^oing outside their circle to see the prob- 
lems that occur in practice. 

Many people have been and are still misled by the "classical experi- 
mental paradigm". For example, an experimenter wishes to test the effect 
of different line lengths and controls for size of type, -interline spacing, 
and so on. Results are obtained and recorded in all the textbooks. But, 
altogether unnoticed, the researcher has used justified lines; that is, 
f-ipacing between words varied so that the right-hand margin is kept aligned. 
Now, uhis interacts with the length of line: the shorter the line, the 
rnore the interword spaces vary; vrlth lines five oi six words long, some of 
the spacing effects can be quite bizarre. Very likely, such spacing effects 
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do bias line length tests in favour of longer lines. Yet, in all the 
classical line length tests,' justification was not "noticed" and hence, 
not controlled. There is actually now some evidence that ranged left 
(unjustified) settings are more legible for short lines and less able 
readers (Poulton & Gregory, 1970). This is small comfort, for without 
Redoing dozens of earlier experiments, we cannot really be sure how to 
interpret all the earlier results. 

We attribute such happenings not to the incompetence of researchers, 
but to their mistaken use of the classical experimental paradigm drawn 
from the physical sciences. Soma researchers in recent years have tried 
to avoid this kind, of problem by using the analysis of variance methods. 
They treat Anovar ' like a kind of trawl-net bringing in'all the little 
fishes. The little ones get slung out and the bigger ones published. 
There is not much evidence that the advent of thesf modern statistical 
techniques has improved the quality of research; i.i some cases it has 
made it worse by putting a greater distance between the question the 
research worker poses and -the answers he obtai; 

When researchers go wrong, they go wrong irj a big way. Empirical 
research is difficult, expensive, and time consuming. At any given time 
there will only be a few people conducting experiments into any particu- 
lar problem area. If they misdirect their attention, many.years may 
elapse before more fruitful work can be started. 

We have proposed a model for researchers that we think is more fruit- 
ful than the classical paradigm (see Figure 1). Our model requires them 
to make two preliminary readjustments. The first is to switch the object 
of research from discovering universal truths to improving specific de- 
cisions in particular practical situations. For example, the purpose of 
legibility research we see as to improve the quality of practical deci- 
sions^ by typographers and editors. There are excellent models for re- 
search of this kind: operational research and ergononcfites are two examples. 
The second readjustment is to acknowledge the importance of tacit know-how. 
Any acquaintance with typographers and designers xd.ll show that their 
skills are backed by tacit knowledge which is the product of experience, 
rather than e.r.pirical tests or ''book learning". Usually, researchers 
dismiss such personal knowledge or even deny its existence simply because 
it has not been developed by orthodox experimental methods. But actually, 
as Polanyi (1962) has shown, all scientific knowledge (indeed all human 
knowledge) has its roots in the "tacit dimension". So. we should value the 
personal skills of typographers and designers and take them as the start- 
ing poini; for more fruitful legibility research. 

Having made the preliminary readjustment, I turn nov to the model. 
The model starts with the act; of criticism. Criticism is a lynchpin for 
research, and it is one of the ways tacit know-how gets placed in the pub- 
lic domain. For example, when a designer writes a report for a cleint, 
he usually starts with a critique of existing solutions. Few problems 
are really novel and he can start by pointing out the strengths and 
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?.aknesses of previous designs, and thus,, set up criteria for his own 
alternative solutions* Criticism is a basic intellectual skill. It is 
a primary method in the humanities and fine arts and, as Karl Popper has 
sho^i/n, plays an important role in the natural sciences. The habit of 
^criticism or intellectual discernment is not only respectable, but actu- 
"ally essential for progress to be made. 

The next part of the model is the creation of alternative solutions • 
These alten^atives can be subjected to criticism and testing. The altern- 
atives make use of the tacit knowledge which researchers have previously^^ 
tended to disdain. 

The third part of the model introduces empirical testing. Of course, 
there is a limit to the improvements that can be brought about without 
testing, but tests are not the first resort of the practical man. By the 
previous stages of criticism and alternatives, we ensure that the testing 
is done on a real life version of some high quality, with all the complex- 
ity that entails. This means we are closer" to formative evaluation or 
industrial production testing than to the classical paradigm. And we make 
no apologies for this. In our model, tests are not tests of individual 
variables and they are not tests of universals. Instead, the KQJt^tOJidiQA 
tUtti one. zohOAZYVt oLteAncuUvz agciiyist andthoji. These alternatives are 
the refined produce of the critical activity described earlier. In this 
way, the tacit know-how of the designer or typographer is linked to the 
activity of experimental testing. The purpose of the testing is to make 
better decisions, not (directly) to discover universal truths. 

To develop these ideas further, we next looked at another kind of 
visual communication, the presentation of quantitative data in tables, 
graphs, and charts (Macdonald-Ross , 1976c). 

The experiments on pictorial charts by Vernon (1946, 1950, 1951, 
1952a, 1952b, 1953, 1960) shox^ some typical problems in experimental de- 
sign. Vernon tested pictorial charts (which she called "Isotype'') against 
tables and graphs, obtaining a no significant difference result. Her own 
conclusions show that she was prepared to take this result at its face 
value: 

No all-round advantage is gained by using pictorial charts 
rather than graphs. The charts appeal more to the less educated 
individual while the graphs may intimidate him; but it is un- 
likely that he will understand the charts sufficiently well to 
make it worth using them. (Vernon, 1952a.) 

It had been supposed that because these charts somewhat 
resembled ordinary pictures, therefore, anyone who studied them 
would be able to perceive and understand them, not as collections 
of little men, etc., but as data on the frequencies with which 
people performed the activities shown in the chart. It is true 
that, particularly when they first appeared, the novelty of these 
• charts aroused people's curiosity, and therefore they possibly 
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devoted more attention to them than they would have given to 
conventional diagrams or tables of figures. But, as the auth- 
or found in her inquiries, a mere study of the charts and per- 
ception of the pictorial shapes they showed was no guarantee 
that thGir meaning would be understood. Such unde^stand^ng 
necessitates a reasonable intelligence and special training 
in how to interpret them. (Vernon, 1962.) 

-However, there is a perfectly simple alternative explanation for 
/p'^T^, ^^^^^ l^J^ experiments were conducted with Zncompe.- 

t(LyUZy du^n^d g^pkcc aUzAruvUvU. A test of a well-set table against 
a poor graph and a dreadful pictogram is invalid; one is 'not entitled to 
':-^'!,f'^J^''^''^^^°^^^^^i-°^s whatsoever. Moreover, when subjects fail, 
this because only bright, well-trained people can understand pi-to- 
grams? Or because it takes unusual intelligence to successfully unravel 
the visual confusion of a badly designed chart? 

By good fortune, Marie Neurath (one oF the original members of the 
^li^^ ^^11 ^nd living in London. Mme. Neurath 

has worked on pictorial charts for over half a Century, so it seems fair 
to regard her as a I'master performer". With her help we were able to 
ihllV'T.n K S^^^ine Isotype chart from the original data used by Vernon. 
There can be no doubt that the Neurath chart is different from the Vernon 
chart and clearly superior as a piece of graphic communication (see Fig- 
ures ^, J, and 4). Of course, only detailed empirical studies can tell 
us the extent of this superiority - and in due course I hope to report 
the results of such studies. ' f f ^ 

To sum up, it is clear that the chief mistake Vernon made was to dis- 
^^f^r^K^ f ^ °^ ''^^''^ construction. Her examples were simply not worth 
the bother of testing; they would have been greatly improved by the criti- 
cism and alternatives which our model prescribes. The results of her work 
if meaningful at all, say to us only that badly constructed material does ' 
not communicate - and we hardly need researchers to tell us that' She, 
however, interprets her subjects' mistakes as due to their lack of intel- 
ligence and prior training. (Taus , is research so often contemptuous of 
ordinary people whilst being rhoruvighly uncritical itself.) 

The passage of time has, alas, taught researchers very little and 
they continue daily to make the same kind of mistakes. As a result, im- 
portant problems are not sufficiently researched, and when they are re- 
searched, the results are often not robust enough to stand up to applica- 
tion in the field. Yet I continue to believe that research is necessary 
and that these problems can be overcome. 

'm 

From all these criticisms one can make a few exceptions. The field 
of readability research is noticeably more organised and more reliable 
than most others. Because it tackles problems which are derived from 
practice, and not from abstract theory, readability researchers do not 
usually have high status in the profession of psychology and may well not 
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(*) These figures, and also the total column, exclude prisoners and missing. 

(^) Munitions industries, i.e.. Iron and steel, non-ferrous metals, shipbuilding, engineering, 
aircraft and vehicles, instruments, chemicals, expl wives, etc. 

(') Agriculture, mining. National and Local Government, transport, shipping (including 
Merchant Navy), public utilities, food manufacture. 

i*) Building, textiles, clothing, distribution, professional services, etc. 
Schoolboys, students, invalids (iticluding war invalids), retired, etc 

, 13. Table i shows that while the numbers in the Armed Forces, whole-time 
Civil Defence and the munitions industries have increased, the number em- 
ployed in other industries has been reduced. In 1944 the number of men 
engaged in the Group III industries was bialf of the corresponding pre-war 
total. 



Figure 2. Source of Data for Vernon's Chart. (From the 1946 
British Government White Paper, StatUticji 
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Figure 3. Vernon's Chart. 
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How men were employed during World V^ar II 
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Figure 4. Neurath's Chart Based on Same Data. 
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get the credic they truly deserve.' I do not say this simply because at 

this conferen ce there ctre sonle of the very workers who have contributed 

to this field of research, but because it is my truly held conviction 

that other applied psychologists could learn how to. redirect their efforts 

by close observation of this successful field. 
.A 

The Transformer . 

Like most organisations of our age, the Open University has a pro- 
duction-line system. Many different people help create the teaching 
material: authors, educational tecimologists , editors, der:igners, illus- 
trators, photographers, television producers . . . All^these work in 
separate departments, often quite uncoordinated one with another, and 
all too often mutually suspicious rather than cooperative. No one person 
is responsible -for mediating the whole creative process* With so many 
fingers in the pie, it follows (of course) that AjnpohXjiy^J: tlvLng^ a/it lz{^t 
undom and no-onz acczpts /i(ZJ>poyi6lbAZ<Xy {^o/i thz ^accQ/j6 thz toxU (U 
JJUtAmzyit^ ComauyiLcxvUon. In my opinion, the University works not 
so much because we are expert at distance teaching, but because the stu- 
dents .are ke«i^n, intelligent, and hardworking. Jne cannot always rely on 
such a response. 

X suggest such large organisations lack a crucial process — a'AOM- 
fjOA}r.aJ:A.oyi — and a person to carry the process out ~ the iAayh^i {^chmz/i. 
He is the skilled professional communicator xv^ho mediates betweer. the ex- 
pert and the reader. His job is to put the expert's message in a form 
the reader can understand, and to look after the reader's interest in 
general. For example, any reasonable query the reader might have should 
be thought about and catered for in a proper manner. Put this way, the 
role seems simple, but it is not. A transformer needs a good general 
education and a wide range of particular technical skills. A transformer 
will find himself dealing with all kinds of experts, most of whom do not 
know what they want to say, or how to say it; he will discover how little 
is really knora about the readers, and the techniques he needs are scat- 
tered 'around in the most: unlikely places. 

.1 would like to have you think that the idea of the transformer is 
my ox^m, freshly minted. However, it is not. It v/as first coined by Otto. 
Neurath when he developed the system of graphic communication known as 
Isotype. Tliough I extend the idea beyond its original scope, Neurath' s 
basic idea was correct, important, and . . . ignored for half a century! 
There are penalties for being ahead of your time. 

The transformer starts with what to say, and then resolves how best 
to say it. Naturally, this distinction must not be overdone. What you 
want to say does partly determine how you say it and, in return, the con- 
tent of a message is always altered to some extent by the way it is put 
over. Nevertheless, the distinction is a useful one. 
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First, one discusses the CJOYltZYVt of the message with the experts; 
later, one works out the exact ^om of the message with the help of illus- 
trators, photographers, printers, and other technical staff. The skills 
needed and the problems faced differ at' each stage. Finally, the cycle 
is complete when the transformer discovers vhat e|J|{eC/t the message has 
on the reader . 

The transformer is overseer of the whole process of communication — 
what is said, how it is said, and what its effect is. He works with col- 
leagues whose skills are more specialised to make sure the message gets 
across and to reduce the change of communication breakdowns. He acts on 
the reader's behalf as best he can, sorting out the kind df issues a read^ 
er might raise if he were present in person. 

VJhat To Say 

The transformer must be as closely involved with the content of the 

xnessags as he is later with its presentation. He must understand the sub- 
ject at hand so that his later judgements are in{^0^"m\zd judgements; but 

there is more to it than that. The transformer will usually need to help 

the expert get his ideas sorted out so that the subsequent act of commu- 
nication has some chance of being successful. 

The transformer is the partner of the subject-matter expert and not 
his slave. Our experience suggests that experts are not always as expert 
as they seem at first sight, and even published statistical data may be 
unreliable or misleading. Moreover, the expert's natural tendency is to 
think of his subject, rather than to think of the reader." For reasons 
of this kind, the transformer .should not accept an author's instructions 
without cricical thought. He must question and analyse until he can put 
the author's intentions in proper perspective. The idealistic young com- 
municator may be surprised to find his source vague and confused instead 
of clear and authoritative, but the man is only human! He may not have 
organized his ideas yet; he may feel uncertain and anxious about issues 
that he has not sorted out but which cannot be ignored. The transformer 
helps as a sjmipathetic listener who gently refuses to go away until the 
confusion is sorted out. Any any rate, he must stick to it, for no-one 
can make a good comriunication out of muddled thinking. 

The transformer sets about his task with the help of two insights: 
a good communication is ^oZzcJiZd f^O/i a puJipO^d and has a 60and logical 
^tAiXCJtuAil. These two insights lead to the techniques which a transformer 
can use in his discussions with the source. 

« Purpose and Objectives. All human communication depends on 
artful selection, since one can never say all that is known. The simplest 
basis for selection is a clear statement of purpose. Ilia dialogue between 
expert and trans foTnr.er may start with vague statements like: the reader 
should have some idea of this or should appreciatie that. The transformer 
works to derive objectives which are as precise as the particular situa- 
tion requires. For example, he may ask, l\}ka;t lOiZl a lLLaaZ66{^uL ouJldOmz 
look tiko. ? What will the reader be able to do if he has interpreted the 
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message correctly? After clarifying what the expert wants to say and 
why he thinks it worth saying, one can discard irrelevant material and 
identify the prospective pay-off for a prospective reader. 

« Tasks and Errors. Sometimes the work on the objectives must 
i^e supported by the kind of professional study often found in military 
or industrial training. If the reader is to perform a well-specified 
function or activity, then that activity can be analysed into its constit- 
uent tasks, and typical mistakes can be collected and analysed. Often, 
such a study starts with the "master performer" - an experienced person 
who exhibits all the skills and know-how necessary to meet the most de- 
manding job standards. Such a study justifies the selection of objectives 
by connzating tkz mz66agz uUXk ikz /izadz^U mnJLd. 

• Organising Principles. In education we make less -use of the 
notion of tasks and errors and more use of organising principles. All 
subjects consist of facts, arguments, theories, problems, and procedures 
which (to a greater or lesser extent) are unified by central themes or 
organising principles. Sometimes these principles are quite grand, cover- 
ing vast domains (the theory of evolution would-be one such example). 
Sometimes the principles are more practical; for instance, computers can 
only carry cut those operations which can be exactly specified in a pre- 
determined code. Sometimes (as in the social and human sciences) one 
finds competing world-views which one must understand in order to interp- 
ret the opinions of the experts. Such world-views are to some extent 
mutually ZxcZiU>lvz and a transformer must realise that they lie behind 
and largely determine what experts say and do. The principles help the 
expert, and can help the transformer, organise a map of the terrain. 

• Facts. It is a fact that 8% of men (but only 0.4% of women) 
have defective colour vision. It is a {^act because experts agree on what 
"defective colour vision" means: there are standard tests to identify it 
and the results of key investigations are not in dispute. On the other 
hand, if someone says "whites are more intelligent than blacks", the status 
of such a statement is much less clear. The statement is not kno\m to be 
true; but it not known to be untrue - it is in dispute. Therefore, 
the . job of a tra'.:r : )rmer is to intercept opinions or interpretations mas- 
querading as factr. He must identify the ^toutsjub of any key statements 
made by the expe^^t. 

o Truth ati-'', Validity. People differ in their ability to deploy 
arguments in an appropriate manner. This is too big a subject to be fully 
treated here, but so important that it must be mentioned. Th3 transformer's 
dream is to see all arguments valid and all propositions as they claim to 
(true, untrue, or uncertain as the case may be).. Although this is the 
kind of quest which never quite ends, even a moderate amount of success 
does a good deal to ease the reader's problems. 
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tor h«lp und«r th« provlalao* of Oi« tfatlonal Uvalth ^rvloa 
Act lV4ti, th« H««lth i)«rTlc«3 and JMoUc H«Alth Act I9b8. 
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a vonlcle to get to ana iro« iull-tloe, or alooat full-tiae, paia wortt. (Ii' your walkliw; ability 1» restricted to tne ertent 
th.1t you would ccKie wltnin thla tnird catejrory out yoc are not ewployed you may be conaioered ior the supply of a venlcl* 
on dooaatio >jro\mdB if one la essential for tne Independent iunotlonln«f of your houaerwld^J 
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Do you 
need a 
vehicl 
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You arc entitled to 
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oi vehicle excise duty. 



Tou are eligiole ror .i 
car with iree insurance 
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duty. 
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veniciti excise 
It •>einj; used by 
> If desired* an 
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the priv.ite c;»r 
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You are not 
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Figure 5. An Algorithm Bef.ore Transform! 
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TRAWSPORT FOR THE DISABLED 
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Figure 6. An Algorithm After Transforming. 
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Bias, Most people .welcome a second opinion to help get the < 
hf.Ki:\,. j^} of an article .right, but thoroughgoing deliberate bias i3 diffi- 
cui.\ "^-o cure. The transformer may be resisted because tha author, or 
even ^'.i.e whole organisation, has a vested interer?t in -Ti-iintaining the 
bias. Companies may wish to hide defects in their products or in their 
finances; government departments may not wish the public to know their 
full rights under the law (see Figures 5 and 6) ; lecturers nowadays do 
often allow their political opinions to colour their courses* In such 
cases, the trcinsformer may get caught by his dual loyalties. If he crav- 
enly submits to his colleagues, then he is just a lackey (Running Dog); 
if he sides entirely with the reader, then maybe he cannot influence his 
own organisation (no teeth — Paper Tiger!), Tl-.e transformer should 
play for honesty and integrity: qualities which do tend to pay off in 
the .long run. .Otto Neurath used to say "the transiiormer is- the trustee 
of the public" ' — that is a pretty fair way to put it. 

How To Say It 

The transformer's job now is to plan tne plU^yvtcution of the message. 
Since text can only use printed language and graphic devices to put over 
the message, to do this he needs skill as a writer and editor, and as a 
^yp'^grapher emd graphic communicator. Of course, he does not have to 
write or draw everything himself! — but he must be skillful enough to 
ensure that the presentation enhances the message, 

o Language, No transformer can hope to succeed unless he is a 
skilled wordsmith. Experience and experiment teach us that simple sen- 
tences \<r±th active verbs and familiar words can be read and understood 
by most adults. As the words get less familiar and the construction gets 
more complex, so more and more readers fall by the wayside. Different 
target populations have different levels of reading competence, and it is 
the job of the transformer to know his audience, to make sure the language 
is tailor-made to their requirements. Reading tests can be used to assess 
the competence of the target population^ readability measures can be used 
to predict the difficulty of the text, and books which show how to xrnlte 
clear English can' be consulted, 

e Links, New or difficult ideas can be grasped by the reader, 
but only if one makes* the right connexions with his familiar v/orld. By 
his explanations and choice of examples, the transformer can make diffi- 
cult ideas seem easy because they are Z^nko^d to things the reader already 
understands. In this way, good WTs'iing makes use of a reader^ s own knowl- 
edge and experience. Technical terras can, if necessary, be put so that 
we can all understand, yet at uhe sar ■ time, the transformer must not de- 
base the original notion by sloppy t[iin.king or unclear expression. Expla- 
nations can and should ]:e checked witli the expert, and reference books 
consulted. 
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• Typography and Graphics » To attract and keep interest and to 
ensure legibility arti the classic aims of the designer and the typographer. 
These aims are still iiaportant, but the transformer must tiave a much broad- 
er vision than that. Often, the graphic elements carry a crucial part of 
the message, so one cannot regard design as just external decoration ap- 
plied to an existing message. Rather, we go along with the ideals of the 
New Typographers: design is an integral part of the communication process. 
Good typography helps the reader plan his reading strategy, tells him 
where he is and helps him to find his way about; good graphic design al- 
lows one to - 'V in words and illustrations what could not be said in either 
form alone - 

« 

o Ac -1 . If one wants the reader to do something after reading 
the text, r points are crucial. We assume that Mhcit the reader 

should d. 1.; r i -^ady decided (by task analysis or whatever) now we want 
to Icnc ! --:t enables him to perform. The three key questions are 

^'i /^cand what we are asking him to do? Can he actually do it? 

^ and psychologically possible?). Will he know when he has 

completed ..lie task successfully? The simple Highway Code rule ''RED means 
^top'' meets all three criteria simultaneously in' three words. iilrut, what 
about "at 70 mph shortest stopping distance is 70 ft thinking distance + 
245 ft braking distance = overall stopping distance 315 ft"? This is use- 
less information. In the first place, the accuracy is spurious, ao \he 
Code admits (wet and slippery roads, different vehicle^, poor brakes and 
tyres, tired drivers . . .). But, in any event, how on earth are drivers, 
meant to translate those nvirabers into appropriate action on the road? The 
answer is, they can't and they don't! Good drivers may learn to judge 
stopping distance reasonably well, but no thanks to the Highway Code for 
that. 

• Organisers and Signposts. The old saying goes: "First I tells 
them what I'm going to tell them; then I tells them; then I tells them 
what I|ve told themi" People do need to know where they are, where they 
are going, and what the prospective pay-off is. As we proceed, they need - 
to know what the ^tatu^ tliQ, me^^iage is. Are we giving them main or 
subsidiary information? Are v/e asking a question or giving an instruction? 
The form of the message must show its status and function. Texts are full 
of devices which help the reader find his way around; typical examples are 
objectives, prefaces, introductions, contents lists, headings, questions, 
instructions, numbering systems, glossaries, and indices. If thecommuni- 
cation aims at specific goals, then the organisers are strongly directive 
(objectives, instructioni) ; if the reader is allowed more freedom, then 
the permission-giving organisers becomes more important (contents, head- 
ings). Thus, the choice and emphasis depends on the purpose of the com- 
munication and the situation of the reader. Organisers are not isolated 
bits and pieces; they must fit together. The contents page, glossary, 
index, numbering system, headings, diagrams, and main text are mutually 
interdependent; what one "does in one place affects all the others. The ^ 
design and layout of a two-page spread can become a little cameo of trans- 
formation; all kinds of informed guesses are made about reading behavior 
so that the reader can best access the information he needs. 
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9 Feedback. Since we cannot be there to answer questions, tha 
reader nay often wonder whether he has understood us properly. So we 
must allow him to check his understanding by giving him feedback. We 
must tell him how to recognise when he has executed an instruction or 
^answered a question correctly or understood a key idea. The feedback 
should debug typical errors (take care of the most frequent kinds of 
mistake) and should be placed immediately after the stiiaulus (question, 
instructior., or whatever) . It is worth mentioning that feedback can be 
quite subtle; for example, repeating the same idea in different form 
gives the reader a second map-bearing which checks his first interpre- 
tation — *'0f making many books there is no end; and muqfi study is 
weariness of the flesh" (Ecclesiastes 12, v 12). 

« Tesr.ing. The transformer cannot get everything right; he is 
lucky to get thing:3 half-right at first. Yet the final version may need 
to be very ^ood indeed > if it is to succeed. How do we face this prob- 
lem? We say, there is a time for guessing ^d <X timd ^Ofi tU-tbiQ. The 
testing shoulL. be done during the transforming process on a rough but 
complete draft version. The object is (obviously) to gauge whether the 
purpose of the conmiunication is being achieved, and to identify and col- 
lect sources of misunderstanding. Any text which is important enough to 
have money spr-it on it is important enough to be tested, and the trans- 
forming process must allox^; time for the lessons learnt to be put into 
effect. 

The Later S t ages 

Earlier we saw how the transformer worked wiLh the expert source; 
now we see him working xr/ith t\<!0 quite different kinds of persons — the 
skilled craftsman (or technician) and the research worker. We deal here 
in less detail than earlier, though in the long run the tranr.f ormer ' s 
relations with craftsmen and researchers will be just as important as 
his relations with sources. 

o Production. The transformer depends upon illustrators, photog- 
raphers, compositors, film editors, and other skilled persons to help him 
realise his vision. These people can make a real contribution. At their 
best, they can teach the ' trans former a good deal about the possibilities 
and limits of their arts, and about the. costs and implications of various 
production methods. In some cases (scientific photography, for example) 
a skilled person can help the growth of \the subject-matter as well as its 
subsequent presentation. The transformer should deal with these people 
in person, not through any kind of intermediary. It is worth mentioning' 
that without the transformer, the final product often reflects the pro- 
duction process rather than the needs of the reader. How often have we 
seen the fussy house-styl'^ applied unintelligently or the diagram which 
has lost its meaning because the illustrator did not understand it! 
Often a free-hand sketch by someone who understands works better than that 
same sketch redrawn "professionally** by an illustrator. Underground 
papers use modem print methods to bni,nQ th.Z iZOimunicicuto^ qImqJi to tkz . 
/i2adey\. We should think long and hard about such ideas. 
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• Evaluation. "Discovering the effects" is the last part of the 
transforming cycle. This means going out into the field and looking at 
aXJL the consequences of the communicatibn that can be found. The really 
gross errors should be picked up earlier in developmental testing, but 
many subtle effects only show up later in the field. The information 
gained can be used for subsequent editions, but is in any event useful, 
since similar problems and situations are just bo<and to occur. Trans- 
formation is an active process that learns, not a passive dispensing of 
recipes. Evaluation can be a complex technical matter. Unless the trans- 
former himself has research training, he may well need the services of a 
professional researcher to alert him to ways of collecting data. The 
budget for evaluation depends on the cost and importance of the communi- 
cation and the prospects of similar publications in the future; we do 
suggest that there should always be such an item in the budget. 

• Recurrent Problems. As time goes on, the transformer will 
notice certain problems coming up time after time. Sometimes they cannot 
he solved simply or intuitively; they need detailed work in depth for a 
long time. Tnis is the job of the basic researcher. The relationship 
between these two colleagues Is often uneasy, with thi transformer want- 
mp solutions yesterday and the --'searcher limited by what it is p044ib£e 
to do, r;iven present knowledge :a resources. A similar case is the rela- 
tion between general practition^ . and medical researchers. The GP has 
patients^ now in his surgery, but the researcher knows that some questions 
just don t have easy answers. For all that, the art of medicine advances 
by this mixture of cooperation and conflict, and so should the art of 
communication. 
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The views expressed in these papers are those 
of the authors and do not necessarily reflect 
the views of their respective military services. 
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INTRODUCTION 



The current research and 'development programs in the areas of 
reading readability, and useability in the Armed Services hav£ built 
upon earlier. efforts in these areas. This section looks back on that 
earlier R&D to help provide a perspective on the current R&D. 

In the first paper of the section, Dexter Fletcher highlights 
developmental efforts in his historical survey of. literacy training 
in the Navy. Looking at the years from the 1700s to the outbreak of 
the Second World War, he discusses the early literacy training, which 
was prompted in part by religious reform and in part by the naval 
apprentice system. He then presents a more extensive discussion of 
.ne three literacy training programs of the period following 1943: 
tne Special Training .Program, which. was prompted by World War II- the 
Recruit Preparatory Training, which was prompted by the Korean Conflict- 
and Academic Remedial Training, which was prompted by Vietnam and 
. Project 2.00,000. 

Fletcher emphasizes the need for systematic literacy 'research and 
development, rather than periodic development in times o'f emergency 
which has been the case in the past. He aLso points to the increasing 
concern for job functional literacy to enable Navy personnel to perform 
their jobs. 

The emphasis switches from development to research as Jarklliller 
surveys literacy research in the Army. He uses a systems approach to 
uncover pertinent research, and the; lack of it, ia the areas of reading 
training, technical .siting, and useability of job and training i^.ateri- 
^.l.s. His uistorical review points up the fact that research has lagged 
rar behma development in reading and readability R&D in the armnd 
services. 

_ The history of literacy research and development -'n the Air Force 
ot necessity begins .later than that of the other serv .r. In present- 
ing that history, Ron Burkett reminds us that the problem has always 
been a. two-sided one of both readability of the materials and reading 
ability of the personnel. He discusses both sides of this problem 
On the one side, he presents the efforts to determine the readability 
of uir Force publications, both job and training materials, and the 
attempt. Co make those materials more readable. On the other c^ide he 
..ooks at studies which attempt to determine the reading abilities of 
Air Force personnel and those which document the match or mismatch be- 
tween the reading .-^hility of personnel and the readability of the matp- 
rials. Tne documenterl mismatch, or literacy gap, in 10 problem career 
tieids IS approximately two reading grade levels. 

Burkett also looks at some of the ways in which the Mr Force ha« 
dealt with the problem. He reports on personnel studies, which have " 
been used to support a .policy of minimizing the literacy problem through 
high selective standards;' evaluation studies of the literacy trainin" 
prograir.s, which attempted to close the literacy gap by raising the read- 
ing abilities of the personnel; and developmental efforts to produce the 
job performance aids, which attempt to lower the reading difficulty and 
increase the useability of job materials. 
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HISTORICAL PERSPECTIVE ON LITERACY TRAINING IN THE NAVY 

J.D. Fletcher 
Navy Personnel Research and Development Center 

History seems to be aot jo r.uch a record of what has happened as a 
record of what we remember, and any historical perspective is most prob- 
ably a perspective cn perspectives. This is certainly true of the current 
historical perspective on literacy training in the Navy. Because of the 
subjective nature of the data that support it, this perspective is divided 
into two parts: a very b/rief part dealing with all of Navy history prior 
to 1543 and a more extensive part beginning in 1943 and ending with the 
present. Further, very little information is presented on readability 
and technical writing. Historical information on these topics doubtless 
exists, but it is buried in the instructions and official correspondence 
of the Nav\' Department, ' 

A comprehensive, thoroughly satisfactory definition of literacy train- 
ing was not attempted in this report, A rough description of literacy 
training as an attempt to bring personnel who read below the 4,0 grade 
level up to that level appears to serve fairly well the needs and scope 
of this report. In the history of Navy-^ training , this description empha- 
sizes training for enlisted men who are in their first tour of duty, 

A. Literacy Training Before 1943 

Enlisted personnel are, at best, neglected in military histories. 
Battles, tactics, and technologies tend to be recorded in some detail, but 
the ability and character of the enlisted force that are basic to any mil- 
itary service are often obscured by the statistics of manpower supply, 
loss, and demand. If it is reasonable to assume that enlisted personnel 
acquire/a greater significance as the technological demand of their duties 
increase, then" their neglect in histories of the United States Navy is 
particularly regrettable. It is difficult to read any account of enlisted 
life vd-thout bringing away an impression that Navy duties have always de- 
manded substantial technological capabilities. The introduction of steam 
and electrical systems aboard ships obviously increased the technological 
demands of enlisted Navy jobs as Cummings (1929), Harrod (1973), Potter 
(1913), and others have indicated, but discussions such as those of Luce 
(1890) and Niblack (1891) of training problems that existed in the earlier, 
sailing fleet indicate that substantially more than disciplined responses 
to orders and knowledge of nautical terminology were relevant objectives 
in transforming "landsmen" into seamen. Most of this training was accom- 
plished on the job following the apprentice, journeyman, master craftsman 
model. It required listening but not reading skills, and it continued 
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throughout a sailor's career* Despite the predominance of this mode of 
training there were significant efforts as early as the first half of the 
19th century to provide instruction in the basic skills of reading, writ- 
ing, and arithmetic to Na^/y personnel. . There was no land-based training 
establishment at this time and basic skills training took place entirely 
atoard ship, at dockside, and while the ship was underway. There appear 
to be two primary reasons for the early literacy training: the religious 
reform movements of the 1700s and the assigmut^Jit of teen-age boys to the 
fleet. 

1 . Religious Refonn 

In the late seventeenth century and during much cf the eighteenth 
century, a major transformation in Western civilization took place under 
the auspices of -the Enlightenment. Leaders of this movement 'placed great 
faith in human reason backed by the findings of science and submitted to 
question all authority and absolute standards. As a result, a sympathetic 
and humanitarian outlook on the condition of all men was stimulated, 
especially by rel:i,:-ious manifestations of this movement* Protestantism 
particularly fo:-?rt ced an enlightened self-interest on the part of its fol- 
lowers by emphasising good works and the notion that every man is his 
brother's keeper. Heaven would be won and past wrongs atoned for, if each 
man would firmly embrace religion and help his fellow man, especially his 
less fortunate fellow man, to do the same. Some members of the evangel- 
ical m.ove-ent, which grew naturally from the Enlightenment, chose to focus 
their interest on the sailors of the naval and merchant service. Revivals 
were held, Sunday schools were conducted, and tracts viere distributed. 
X-;ith the distribution of the tracts came the realization that the word of 
God was inaccessible to many sailors because they could not read. Moreover, 
sailors' moral welfare appeared endangered because their lack of aritlime- 
tic skills made them easy prey for the peddlers of prurience who were nat- 
urally dra^^m to the full pocketbooks and poorly educated minds of newly 
disembarked sailors. So it was that the first to minister to the need for 
essential skills training among Navy enlisted men were the early Navy 
chaplains (Langley ,, 1967) , 

The regulation of 1802 described the Chaplain's duties as the follow- 

1 Fe is to read prayers at stated periods; perform all funeral 
ceremonies over such persons as may die in the service, in the 
vessel to which he belongs; or, if directed by the commanding 
officer, over any person that may die in any other public vessel. 

2. He shall perform the duty of a school-master; ^:nd to that 
end he shall instruct the midshipmen and volunteers, in writing, 
arithmetic, and navigation, and in whatsoever may contribute to 
render them proficients. He is likewise to teach the other youths 
of the ship, according to such orders as he shall receive from 
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the captain. He is to be diligent in his office and such as 
are idle must be represented to the captain, who shall take 
due notice thereof. (Burr, 1939, p. 111.) 

In addition to Chaplains, there were schoolmasters and teachers in 
the Navy as evidenced by the law of March 1799 assigning them three- 
twentieths of prize money, which was approximately the amount assigned 
to warrant officers. In general, the schoolmasters made a poor showing. 
Allegations of sloth and drxmkeness on their part appear well founded. 
Schoolmasters^ duties varied \^th commanders, but they were generally 
charged with instructing the boys, apprentices, and midshipmen assigned 
to ships. The employment: of schoolmasters was never extensive in the 
Navy, and it gradually died out. However, the history of Navy chaplains 
is far more honorable than that of the schoolmasters, and the Chaplain 
Corps maintained its interest in essential skills instruction for enlist- 
ed men throughout the history of the Navy (Drury, 1949). There was, of 
course, little systematic method in this instruction, and its success 
most probably keyed on the moral rather than cognitive well-beii. of its 
students. 



2. Apprentices 

From the beginning of the Navy there were boys assigned to ships • 
Attempts to standardize their employment and treatment were embodied in 
the apprentice systems set up in 1837, 1855, 1863, and. 1875, Of all the 
training ventures before the Spanish-American War, only the naval appren- 
tice program provided a prototype for the modem Nav>^ (Harrod, 1973). 
Despite this fact, the apprentice system suffered unpromising beginnings. 
The 1837 effort failed because apprentices' hopes of obtaining coniinissions 
were not fulfilled and because of an alleged mutiny in 1842 on the train- 
ing ship Som^yU, in which a son of the Secretary of War was hanged, prob- 
ably wrongfully (Langley, 1967). The 1855 program was interrupted by the 
outbreak of the Civil War. The 1863 effort was unsuccessful because ap- 
prentices were again disappointed in their hopes of obtaining commissions. 
However, the apprentice system continued to be revived because of very , 
serious problems the Navy was experiencing in manning American ships with 
native-bom seamen (e.g.. Luce, 1974). In 1875 the Navy began enlisting 
boys 14-18 years of age to serve as apprentices until age 21. The Secre- 
tary of the Navy emphasized that it was not the object of the system to 
prepare boys for commissioning; the apprentices were to be trained in all 
the duties of sailors on a nan-of-war, and, significantly, they were to 
receive an elementary English education (Harrod, 1973). Despite numerous 
problems, including the neglect of academic training, the. system, with its 
commitment to literacy training, survived until it was replaced in 1904 by 
an officially established apprentice seaman rating with a minimum age re- 
quirement cf seventeen. With this system evolve.d the prototype for to- 
day ^s recruit training. 
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Recruit training was ess^entially motivated by a need for standard- 
ization in training landsmen "and apprentices for shipboard duty. A nat- 
ural means for standardizing training that was employed very quickly after 
the need for it was recognized was the publication of drill books. Full- 
am s. KzcAiuX'^ Handy Book and McLean's BluiZjaakU^' Uayuxal both appeared 
In 1902; Fullam's book lasted until the 1920s and McLean's manual is cur-- 
rently in its nineteenth edition. By General Order 114 of November 1902, 
the Navy Department required all recruits to know the contents of the 
RecAOct'^ Handy Book and issued a copy to each. Fullam also published in 
1902 the T^y 0 {^f^c^^ ^ V/Ull Book. 

The appearance of these handbooks signaled the fact fhat oral in-- 
struction was no longer sufficient for an enlarged Navy' (cf . , Har:' d, 1973) 
Presumably, thi^ fact was already apparent. Some of the technical equip- . 
raent aboard Navy ships at the beginning of the century .was fairly sophi- 
sticated y and it seems likely that this equipment was accompanied by 
essencial manufacturer's literature on its operation, maintenance, and 
repair. However, there appears to be little record of the nature, prep- 
aration, and supply of this , technical documentation. In any case, the 
appearance of the drill books signified an official expectation that sea- 
men would bs able to read, and literacy effectively became a requisite 
Navy skill. 

Despite a national requirement for literacy, about 11.3 percent of 
the United States population in 1900 could be classified as illiterate 
(Harman, 1970). There were no systematic attempts to screen Navy recruits 
until 1925 (Harrod, 1973), and illiteracy among Navy personnel became 
xncreas5.ngly serious. Although there is no direct reference to this 
effect, tha systematic screening that began in 1925 included written tests, 
successful performance on which required literacy. It seems likely, then, 
that the incidence of illiteracy in the Navy was substantially reduced by 
the use of these tests. 

Aside from the work of the Chaplains, the only formal literacy train- 
ing that appears to have been supported by the Navy in the period 1900- 
1943 was in Portsmouth Prison where some effort was made to rehabilitate 
men who were classified and jailed as deserters because of their inability 
to read furlough orders (Potter, 1918). These men • • .-Id go on leave and 
return on what they thought was the appropriate date only to find them- 
selves scheduled for court-martial. 

Despite the screening process begun in 1925 and official pol ' .ies for 
their exclu:iion, illiterates continued to appear in Navy billets . .rough- 
out the entire period (Special Training Program, 1951). Other than the 
rehabilitation program at Portsmouth Prison, no official efforts were made 
to help these men achieve literacy. There was, and is , considerable un- 
skilled labor required in many fleet billets, and these men were typically 
assigned to do this labor. As might be expected, rates of promotion and 
re-enlistment among these men were not high. During World War II, man- 
power supply began to run substantially short of demand and in June 1943 
the Navy reluctantly agreed to accept its fair share of illiterates under 
Selective Service. 

6n 



71 



B.' Literacy Training in' 1943 and After 



1- The Special Training Program 

Although prior to June 1943, there was no of ficial recognition of the 
Tact that room would have to be made for enlisted personnel who could 
neither read nor write, a great many men in this catego'ry were already 
serving in the Nav^'. Immediately after Fearl Harbor, recruiting stations 
were released from the obligation tc administer the General Classification 
Test (GOT) to applicants and since t:iils regulation remained in abeyance 
u.ictl the start of 1945, many illiterates found their way into the Navy 
through traditional recruiting channels (Special Training Program, 1951) • 
In August 1945, the Director of Training indicated in the following summary 
some of the problems the Navy and illiterates experienced with eachr other. 

(1) At times the period allowed for recruit training was 
contracted by the demands of the service to four or five weeks. 

r these circumstances, the trainee" was obliged to acquire 
rge part of his instruction through readings. - It was found 
tw.LL it took approximately four times as long to train an illit- 
erate to perform an average Navy job as it did one who could read. 

(2) The establishment of a training program which did not depend 
on the use of printed matter would have been both difficult and 
expensive. Experience showed that it was simpler and more eco- 
nomical to teach men to read than to devise materials which did 
not require this knowledge. 

(3) The establishment of a smooth administrative routine was 
grossly complicated by the presence of non-readers. A system 

4 for the rapid handling of records was a virtual impossibility 
where men could not fill out information blanks, pay receipts, 
proficiency slips, allotment cards, et cetera. 

(4) Sufficient education to read safety precautions was 
essential for men working with machinery, high explosives, 
and heavy cargo.. Serious accidents were traced directly to 

men's inability to read warnings and study safety instructions. \ 

\ 

(5) A social barrier of serious implications was found to \ 
exist between literate and illiterate personnel. 

(6) The administrative dualism resulting from putting lit- ■ 
erates and illiterates together caused confusion. Literates 

tended to resent the long oral directions which they had to \ 
listen to for the sake of the illiterates in their ^number. \ 

(7) A very large number of minor disciplinary problems were ^ 
the direct outgrowth of misunderstandings caused by inability 

to read station orders, watch bills, leave and liberty regula- 
tions , and safety instruction^ > 
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(8) An inability to read and write letters constituted 
among illiterates a serious morale problem and consequent 
obstacle to satisfactory adjustment to naval life. It became 
increasingly evident that a knowledge of reading and writing 
helped to overcome a feeling of inferiority and tended to 
develop initiative, aggressiveness, and more willing accep- 
tance to the conditions of military life. 

(Special Training Program, 1951, p. 2-3.) 

Despite this analysis, which was written after the fact, there is 
substantial evidence that authorities were reluct... ,: to aScept responsi- 
Diiity for setting up a special literacy program long after the need for 
such a program became obvious. However, as the situation gradually wors- 
n"!^/ Navy -stumbled into a situation that required recognition of the 
need for a special literacy program. On 30 September 1943, the Naval 
iraming Section in Norfolk, Virginia, reported that illiterates were 
being received from boot camps in increasing numbers. Since all the facil- 
ities at this section .Aiere needed for the destroyer and destroyer escort 
training programs, permission was requested to transfer these personnel to 
EviH^''^? Training Section at Bainbridge, Maryland, for further instruction, 
^.viaently, the permission was granted without due consideration of the 
practical consequences of this decision. The result was that Bainbridge 
, found Itself deluged with illiterates and appealed to the Bureau of Naval 
Personnel for help. The immediate response of. the Bureau was to cancel 
the permission that imd previously been granted to Norfolk. However, as 
a result of these events, the need for special literacy training was brought 
to conmiand attention (Special Training Program, 1951). 

_ The presence of a growing body of illiterates in 'the Navy created a 
training problem that it was totally unprepared to face. Luckily, as a ~ 

In^^'^f H^''^?TQ^^^''■''^^'^ literacy training programs by Fattu, Mech, 

and Standlee (1953) shows, the foundation for solvit; this problem had 

been laid much earlier by the work of the Civilian Conser^/ation Corps (CCC) . 

The CCC was established in March 1933 primarily as a mear.s for provid- 
ing productive employment for young men during the depression, Mthough 
It was administered by the Army throughout its nine-year existence, much 
use was made of professional educators in developing and guiaing the edu- 
cational phases of the program. The CCC education program was secondary 
to the work program, and the literacy program was only a small part of the 
education program. It was created to serve the estimated s:.x percent of 
CCC enrollees who were functionally, illiterate in the camp 2ife situation 
i^°?H P J; ^^^'•'^^t^-^l^^^' the literacy training progra., did oxist as 
an Identifiable entity and it severed as a foundation for ft,.- d^velooment 
lo^^^'^l programs, first by the Army in 1941 and later by rh, Nav^ in 
1944 (Fattu et al. , 1953). ' y y ±u 
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The Amy pr.^ am was di.scussed by Heath (1946), who pointed out that 
there were. three distinct groups of illiterates targeted by the Army pro-' 
grcTu: English-speaking illiterates, non-English speaking' literates , and 
Oriental literates. Training was tailored to the special needs of each 
of these groups. The toaching program for the English-speaking illiter- 
ates passed through f:: - distinct phases. These phases were sequenced 
and a student had to m.i .lar each phase before proceeding to more advanced 
ones. Heath describes the phases as the following: 

1. Consonants and Key Words. Sounds of the consonants were 
tauc^hc in this phase by associating 21 key words with English 
consonants. Phonemes such as /g/ , /c / , /^/ , 7;^/, /0>, and />;/ 
v^rn not taueht. ^ 



Monosyllables and Vowels. Nonsense syllables such, as ter, 
ker, and nub ue.vfi initially taught in this phase in an effort 
to train students in the relationship between orthography and 
r.ound. 

3. Tatroduction of Polysyllables. Students were taught lo 
analyr.e polysyllables into monosyllables. 

A. Sentence Structure nad Word Recognition from Context. 
Knrtlicr instruct:ion in word attack was presented and instruc- 
iA.ov in construci.i.nj.; sentences and learning (recognizing) 
wordij from r.onto/L was added in this phase. 

5. Conpositi.on and Expression. Students W'-r- taught to read 
and prepare military messri^jes and pernon.i^ Ir;, ..-r^- in this 
paar;'-. Tliuy vere al.so taught the use of siu'h i) . ic resources 
as dictionc'iries and teleplione directorie:. . 

niscu.s.->iQn of tiu; Army's literacy program is relt-v'.int because when the Navy 
fjnally faced the need to produce a literacy training progr;mi, it turned 
in ISepteniber 1943 to the Army for guidance, and the workbook material 
developed for the Navy was i)ased on the five phases discussed above. 

On 22 December 1943, the imminent appearance of a special literacy 
prop,ran for recruits was announced by the Na-'ry. On 23 December 1943, the 
Naval Training Section at Great Lakes, Illinois, was directed to prep.ire 
for the arrival on 3 January 1944 of an initial draft of 420 Wliite illit- 
erates. In March 1944, tv/o Navy programs for literacy were established: 
one at^r.'.-unp Peary, Virginia, for Invites, and one at Great Lakes for Blacks. 
Plans for the Crimp Peary program called for a weekly input of up to 500 
crairiees with a total capacity of 6,000. In fact, the total enrollment 
at Camp Peary quickly grew to 10,000 in April 1944 (Specipil Training Pro - 
;;ram,, 1931) . Notably, hotli these programs were set up as an integral part 
or recruit trrrLning; no programs were established for illiterates who were 
already in the Navy. At no time did the Bureau of Personnel formally 
aecepr. responsibility for training illiterates who were above the recruit 
•lev-l, a.lthough informal support was given to Commanders who wislu?d to aid 
•illiterates under their command with the dis..em Inat i(jn oE li.t(jr/icy train- 
ing m/tterials throughout the fleet. 
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The curriculum that was initially devised for the literacy programs 
in early 1944 provided. for only 133 hours of instruction in reading and 
writizig and 73 hours of instruction in arithmetic, for a total of 206 
hours of instruction out of the 576 hours originally called for. However, 
by January 1945, a considerably e:;pajided and improved program bad evolved. 
Four basic assumptions not previously annunciated detemined the form of 
the Special Training Program, as it came to be called. First, it was a 
training program for adults- Although unable to read and write, the 
trainees came to the Navy having command of a well-established oral vocab- 
ulary, together with a fund of experience that put them beyond the appeal 
of grade school readers. Second, the trainees were the products of .a wide 
variety of environments, so that the only interests they could be expected 
to have in. common would grow out of their life in the ser^/ice. Third, the 
limitations on time allowed for acquiring literacy made it mandatory that 
the program be rigidly functional in nature. A nominal proficiency grade 
of 5.0 in reading and writing was establish.jd as the teaching goal, but 
the purpose behind the program was simply to*' qualify ui-^n to read, write, 
and figure sufficiently well to perform all essential Navy duties. Thus, 
it was expected that graduates of the course would be abl<£ to read watch 
lists and safety precautions, and would be able to fill in beneficiary 
slips and small store chits. On the other hand, thcce would be no attempt 
to indoctrinate trainees formally in such refinements as capitalization, 
spelling, and punctuation. Fourth, it was assumed that the vast majority 
of instructors would be inexperienced in the type of teaching expected of 
them. They could be expected to rely unduly on lecture and blackboard 
methods and ::o encourage parrot-like memorization of lists of words with- 
out developing the skills of analysis and synthesis that are essential to 
literacy (Special Training Program, 1951). The CCC literacy materials 
were called the Ca)fvp Ll{^C. series, the Army materials were called the Amy 
L'L^d series, so naturally the workbooks, basic readers, supplementary read- 
ers, feats, and teacher's manual developed for the Navy were called the 
hlavLj LZ^C. series . Private Pete was replaced by Seaman Sam. 

Developnient of the program materials was described by Ross: 

In writing this program, some radical departures from the 
conventional were taken, because the situation and the nature 
of the students and ins t^'uctors demanded them. For example, 
a 'reader" in the hands of an untrained instructor at the out- 
set of the program would result in a static classroom situation 
in which the students "read" orally in rotation, x^ith prompting, 
until the page has been virtually memorized. In the Navy Life 
series, therefore, the first book is not a "reader", but a work- 
book7type text which forces the instructor down from* the platform 
among the students. The first of the readers is not introd \ced 
until considerable reading ability is developed through chart, 
blackboard, and workbook reading experiences. Ijhen it is intro- 
duced, no new skills or words are required for some time, and 
' the studeat can read it easily for meaning. As a result, the 
student is literate so far as the readers are concerned from hl.i; 
very first eixperience vrLth them. 
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Comic books are exceeded in popularity by no other read- 
ing material. So, later in the program, when they can be 
handled easily, comic books are introduced for rapid, super- 
vised classroom supplementary reading, Theae are regular com- 
mercial comic books, carefully selected in advance of publica- 
* tion, and then rewritten with a core vocabulary basic to the 
Navy Life program. 

(Ross C.S., 1946, p. 204.) 

Notably, the con.'e .'stone of this program was not the basal readers, 
which was r robnbly the case in civilian initial reading instruction, but 
the program's two workbooks functionally entitled hlavu Ll^ Book I and 
Navy IxiJe. Book II. Both books were constructed on a basic vocabulary 
that derived firbt from words shown by research (what specific research 
is not in the record) to be an essential minimum for literacy, and then, 
to as great an extent as possible, from naval terminology. Fans of 
Leonard Bloomfield \.all ba interested in Figure 1, which is the first 
"story" that occurs lu Navy L^Z Book I. The emphasis on the single 
spelling pattern '^at" embedded in hat, mat, and sat will seem very famil- 
iar to those acclimated to Bloomfield' s ''Nan can' fan Dar:*' (cf. Bloomfield 
ct Barnhart, 1961). Extensive reliance was placed on illustrations to in- 
dxcats v'ord meaning, but other devices such as a visual acuity test and 
illustrations of phonetic similarity were also used. It was assumed that 
by teaching reading and writing simultaneously, growing proficiency in 
either would increase proficiency in the other. The material in Navy Ll^z 
BoOd I, which contained 400 illustrations, took as its common denominator 
barracks life, which all trainees might be expected to have in coi^mion. 
in the second volume, the emphasis was gradually shifted from phonetic 
e] ents to syllables and from illustrations to context as a means of fur- 
ni .ling clues to word meaning. The rigid control over the material that 
characterized Navy Ll(,z Book I was gradually relaxed in Book II, and was 
finally loosened so that any student capable of handling it would be able 
to conduct independent reading outside the classroom. Navy L^^^e Book 11 
was also prepared wirh the intention that it should be a useful adjunct 
to regular recruit ^ training, since it contained a good deal of incidental 
information on such subjects as semaphore, f iref ighting , elementary navi- 
gation, naval customs, guns, ship types, seamanship, naval terminology, 
and personal hygiene. 

As it evolved, the curriculum was developed on a flexible scale, ' 
operating betx>7een the limits of twelve and twenty weeks. In other words, 
trainees received period!: .ests starting with the eleventh week, and could 
be graduated any time thereafter by showing that they had achieved the re- 
quired level of literacy. The minimum overall time prescribed for the 
course was 256 hours, and the maximum time was 528. Within this framework 
the variation in the allowance for reading and writing ran from 129 to 312 
hours, and for aritlimetic from 63 to 118 hours. 
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The sailor sat on the hat. 



Figu'-e 1. The first story in the Navy ll^t workbook series. " 
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Selection for the program depended on scores achieved on three tests. 

!• The General 'Classification Test (GCT) — a test of general 
ability which in 1945 was re-introduced and given to all incoming person- 
nel. 

2. The Literacy Test (LIT) - a direct meas^-e cf reading achieve- 
ment consisting of 44 items divided into three subtents: word recognition, 
sentence reading, and paragraph interpretation, 

3- The Non-Verbal Classification Test (NVCT) — a reflection of 
thG. GCT administered in noa-verbal terms through the use of pictures and 
geometric forms. The NVCT consisted of 75 items divided fato five sub- 
tests: picture analogies, figure analogies, picture classification (oppo- 
sites), figure classification (opposites) , and matrics (comparable to 
Raven's matrics). 

According to Hodges (1964), the following screening procedure was 
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1. All recruits scoring less than a standard score of 35 on the 
Reading Test (the GCT, when the Reading Test was discarded) of the 
Basic Test Battery were administered the LIT, 

2, Men scoring 40 (equivalent to 5,0 grade level) or better on 
the LIT were considered ^'literate" and sent on to regular recruit 
training, and thos^e scoring below 40 on the LIT were givers fhe 
NVCT. 

3, Men scoring higher than 34 on the mCT were considered train- 
• able, and sent to the Special Training Program. (A raw score of 

34 on the NVCT is between 30 and 35 Standard Score on the GCT,) 

4. Men scoring below 34 on the NVCT \^ere sent to a psychiatric 
unit for closer examination; they were suspected of being both 
illiterate and untrainable, (NB - The LIT and NVCT scores of 
40 and 34 were later changed to 37 and 38 respectively as a re- 
sult of an unidentified study, all records of which have vanished,) 

In other words, the men sent to the special training units were those 
in need of literacy training (low LIT score) and judged capable of assimi- 
lating it (high NVCT secure). 

The Navy Reading Achievement Examination (NRAE) was also developed as 
a standard measure to determine when students in the variable-length 
Special Training Program were ready to graduate. Both the LIT and the 
Nllr\E were calibrated against the Gates Reading Survey to establish grade 
levels. The NRAE was designed to be f u^i.ctional and teste*d students for 
understanding Navy situations in which they might find themselves. 
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„ T^^c^^ appears to be a single evaluation study of tha Navy's World 
War II Special Training Program. This study was administered and docu- 
mented by Hagen and TKomdike (1953) ■ on the basis of personnel records 
salvaged from the Naval Records Management Center, Garden City, New York, 
iwo types of data were abstracted: (1) background facts that might pre- 
,dict success in the Navy, and (2) facts about the man's career that might 
.vf - -! success in Navy duty assignments. Three groups of men were 

1. The "Illiterate" Group. This group consisted of 1026 men 

■ inducted into the Navy during August and September 1944 and 
initially assigned to the Special Training Program a« Camp Peary. 

2. The Control Group. This group was selected by taking a 
serial number that was five digits above the serial number of 

a member of the illiterate group. The man thus identified was ' 

■ included in the control group if he was not black, not illit- 
frate, and came from the same geographical area as his illit- 
erate counterpart. If a serial number five digits higher did 
not Identify a qualified counterpart, one five digits below 
was selected, and so on. In this way a control group of 1021 
men was chosen. This group resembled the illiterate group on 
most geographical and demographic measures. 

3. The Marginal Group. This group was composed of 999 men 
who had scored below 36 on the OCT and who entered the Navy at 
about the same time as the illiterates. This group was geo- 
graphically and demographically dissimilar from both the illit- 
erate and control groups. 

Hagen and Thomdike described the average member of thr-ir illiterate 
group as follows: xxj-Ltirdm 

He was inducted in the Navy when he was about 19 years 
old. At the time of induction, he was single but had one or 
more people who were partially or wholly dependent on him for 
support. Before induction, he had lived in a rural area of 
the South where the standard of living was below the average 
for the nation as a whole. He had completed the fourth grade 
in school and left school at the age of fourteen after having 
repeated at least three grades. Since leaving school, he had 
worked for his parents or a relative on a non-mechanized farm. 
In his spare time he hunted or fished. He made a score of 4 
out of a possible 17 on the Qualifications Test and a score of 
31 on the Navy General Classification Test. 

(Hagen & Thorndike, 1953, p. 18.) 

Hagen and Thomdike summarized the differences they found between the 
Illiterate and control groups as follows: 

1. The illiterates were much more likely than were the control 
cases to be assigned to construction battalions, and were less 
likely to be assigned to U.S. permanent party or to auxiliary ships. 
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The Illiterates tended to receive a lower average pro- 
xicxency in rate.- Only 50% received an average of 3 5 or 
over, as compared with 73% in the control group. 

3. The illiterates received fewer promotions. Only 15% 
made petty officer, as compared with 37% in the control group. 

^llil^^^ates received more disciplinary actions. In the 
Illiterate group, 23% had records of some type of disciplinary 
actxon, as compared with 11% of the control group. General 
cou.ts martial were ten times as frequent in the illiterate 
group. ^ 

H* . "^'"^ m^^^''''^^^ frequently lost time due to miscon- 

duct - 20% versus 7%. 

6. The illiterates less frequently received an honorable 
discharge - 83% versus 88%. 

7. The illiterates were somewhat more likely to receive 

a medication survey, and were the only group to be surveyed 
tor inability to leam - 19% versus 15%. 

8. The illiterates were somewhat more likely to incur a 
venereal infection - 5% versus 3%. 

9. The illiterates were slightly more likely to generate 

a Veterans Administration disability claim - 11% versus 9%. 

°^ that distinguished the illiterate from the 

contxol group the marginal group occupies an intermediate position usu- 
ally nearer rhe illiterate group than the control group. The only excep- 
tion to this IS the case of disciplinary actions; fhe marginal grouj .^eL 
more often m trouble and their offenses were mo^e serious. 

The Hagen and Thomdike study is intersting, but it ±s . s... . . Qy , 
study or personnel at different mental levels. It does not L.vJ.ucn to 
soLtl effectiveness of literacy training. It does, ho.e / " teU 

something aoout the success of illiterates in the Navy, it 
seems reasonable to conclude with the authors that many v 
Illiterates appeared 'to make an acceptable adjustment to '.:he Ma.y,' 

After the war, screening of illiterates with the GC L^T mvcT 
con.inued but the Special Training Program was discontin .w.d" In '-.e course 
of general demobilization. Giuzberg and Bray (1953) estiu,- .,1 hat 35 5oO 
history"'" '° the Navy's Special Training Program . : ..e course of 
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2. Recruit Preparatory Training 

In 1S50, with the infliix of voluntears during the Korean War, com- 
mands were once more authorized and encouraged to identify and tx*o^p. any 
illiterates who might be aboard. By the spring of 1951, literacy rxaining 
\«s being conducted informally in after-hours programs at the ::hV''sa recruit 
training centers (Bainbridge, Marylcind; Great Lakes, Illinois; and San 
Diego, California). By September 1951, literacy training was planned on 
a full-time basis under the new title Recruit Preparatory Ttalrr*.ag (RPT) 
(Standlee, 1954). Officially, the main objective of the pro^,i. ^.'i; was "to 
teach selected recruits to read and understand instructions to pre- 

pare then to absorb military training. Recruits should .at^.air. a level of 
proficiency in reading which is comparable to completion of the fourth 
grade" (Curriculum for Recruit Preparatory Training, 1953, ; 5). Further, 
an accepting classroom atmosphere was to be created as indi.;..'ted by the 
following guidance provided by the Bureau of Personnel. 

1. Make the first reading tasks simple enough to i-isui-o- 
that the recruit experiences an early feeling of success in 
It-arning to read. ' . 

2. Sho^vT the recruit that his low reading achievement is 
a handicap v;hich raust be overcome in reaching his fo-'U — 
becoming a useful sailor. 

3. Create a permissive classroom atmosphere and start where 
the learner is. The individual recruit is the one \.\:v . o 
learn. The most the instructor can do is guide and help i. t 
The instructor should guard against a highly directive, sua 
ject-niatcer-centered approach which is on a level that nevc-r 
reaches some recriiits. 

4. Integration of military subjects with reading, writing, 
and aritlimetic. Somo examples of how this can be done 

] is ted belr/..;: 

a. The ^instructor includes some topics vciJ -i^.-d to 
service in the Navy at appropriate times iu Ms read- 
ing , writing , and arithmetic classes . Thij procedure 
will probably be more effective during the latter part 
uf RPT. 

b. Use o^ visits to ships or parts of the training 
center. Trainees can write about whcit tliey see, read 
accounts to eacLi other, learn naval t'-rvv: , etc. 

c. Learn to read safety sigrs found on the center or 
^bo^ird ship. 

Prcparati.i;! of RPT news sheet and use -.1: local 
n'.:wspapers and .-.innouncenients of rec.'',:eationo 
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e. Provide giiidance in writing letters home about 
barracks, field^ay, bunks, meals, training activities, 
et cetera, 

f . Familiarization with types of material found in 
Tm Blazjadizt^^ ^AaytuaZ, libraries on the center and 
aboard ship, and courses available through USAFI. 

(Curriculum for Recruit Prep- 
atory Training, 1953, p. 11,) 

• The curriculum materials were, vlirh minor updating, composed of the 
Navy llf^Z series prepared for the TVjrJ.J. V.cr II Special TraiLning Program, 
although use of other supplementary u^a^.tirlals was encouraged (Standlee, 
Fattu, Auble, 1954a). The selection procedure for RPT was the follow- 
ing, as desca.ibed by Hodges (1964): 

1. All men scoring below 36 on the GCT were administered 
both the NVCT and LIT. 

2. Men scoring 38 and above on both tests were considered 
literate and trainable and sent to regular recruit training. 

3. Men scoring 37 and below on the LIT (38 on the LIT is 
about a 4.7 grade level) and scoring 38 or higher on the 
NVCT, x^ere considered illiterate but trainable and sent to 
special training. 

4. Men scoring less than 38 on the NVCT and on the LIT were 
suspected to be untrainable and were sent to a neuropsychiatric 
unit for further examination. From the neuropsychiatric unit, 
men were sent to special training, regular recruit training (on 
a trial basis), or discharged. 

Referrals could also be made to the Special Training Program by Compariy 
Commanders in regular recruit training. The NRAE, with a new form added, 
was restored as an end-of-course cv;'. terion. The 5.0 grade level was still 
the course objective. Students were given 7-9 weeks to complete the course, 
An observer who yas' sent to the three RPT centers in 1953 reported that, 
in general, the selection and curriculum procedures in use followed the 
Bureau's directives, that a total of 239 trainees v;ere enrolled in all 
.":hree centers, that 85 to 94 percent of the trainees successfully com- 
pleted the RPT program, and that they took 6-7 to 9 weeks to complete it 
(Fattu, Fay, D'Amico, & Standlee, 1954). 

Two evaluative studies of RPT were completed. A study by Cofer (1954) 
investigated the effectiveness of RPT in achieving its most immediate ob- 
jective, preparing the trainee for regular recruit training. Standlee 
(1954) dealt \-rLth the more ultimate criterion, performance of RPT gradu- 
ates in fleet assignments . 
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A principal concept in Gofer's study is that of adjustment; specif- 
ically, adjustment of RPT graduates to regular recruit training. For the 
purposes of his study, mental health considerations were set aside in 
favor of adjustment that emphasized, first, adequacy of performance of 
recruit duties, and, second, attitudinal and motivational factors in re- 
rationship to the Navy, recruit training, and civilian plans. Fourteen 
rating scales were developed to assess adjustment in the first sense, and 
an 80-item questionnaire was developed to assess adjustment in the second 
Sense. Tv7o groups of subjects were identified; one at an early stage of 
recruit training (Group I), and one at a later stage of recruit training 
(Group III). Within each of these groups, four sub-groups were identi- 
fied. • 
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Each subject was evaluated on his adjustment to regular recruit training 
by peers and Company Commrinders separately on the 14 scales reflecting 
adequecy of performance in recruit training. 

In general, members of the RPT sub-group resembled the members of the 
high GCT sub-grovp on the criterion scales more than they did members of 
the intermediate sub-groups. The results of the peer ratings for Group I 
subjects were the following: 

1. The high GCT sub-group and the RPT sub-group were rated 
significantly , superior to the intermediate sub-groups on per- 
sonal cleanliness, military bearing, bunk and living quarters, 
general Navy performance, and leadership, 

2. The RPT sub-group was rated as significantly superior to 
the other three sub-groups on marching and maneuvers and 
physical drill with rifle, 

3. The high GCT sub-group was rated as significantly superior 
to the other three sub-groups ' on quickness to learn ^nd 
response to orders. 

4. The KPT sub-group was- superior to the intermediate sub- 
groups on care of clothing. 
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The results of Company Commanders ratings of the Group I subjects ware 
the following: 

1. The high GCJI sub-group was significantly superior to the 
other three sub-groups on quickness to learn, response to 
orders, willingness to work, and general Navy performance, 

2. The high GCT sub-group and the RPT sub-group showed superior 
ratings on care of clothing and military bearing- 

The pattern of ratings for Group II was essentially the same as that ob- 
tained for Group I. 

in ge.neral, it seems reasonable to conclude with Cofer that RPT aided 
recruits in adjusting to regular recruit training, but it is still diffi- 
cult to determine what contributions the cognitive aspects of literacy 
training made to this adjustment. 
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Standlee (1954) studied three groups of^recruits in his investiga- 



1- RPT Graduates: . n = 5^1 

2. Marginal recruits with a GCT of 35 or lower, 

an NVCT of 38 or higher, and a LIT of 38 or higher: N = 2,414 

3. Typical recruits with a GCT of 36 or higher: N =■ 1,998 

The 5,312 total subjects were originally selected by identifying all 
RPT graduates and marginal recruits processed during the period 10 May 1952 
to 1 September 1952 and a random 10 percent of the typical recruits proc- 
essed during the same oeriod. Data on the fleet performance of these re-, 
cruits were obtained by mailing out questionnaires to their Commanding 
Officers during July-August 1953. Most of the questionnaires were return- 
ed, of which, 5^023 were suitable for analysis — an effective response 
rate of almost 95 percent. 

The RPT and marginal groups in comparison with the typical group were 
much more often assigned to general duty (unskilled work) and to duties 
that did uot require as much reading skill. The two groups were less fav- 
ored with promotions, had lower average performance evaluations by super- 
visors and Commanding Officers, and had a higher proportion of disciplin- 
ary actions and days lost due to misconduct or sickness. In the opinion 
of division-leading Petty Officers, the two lower groups generallv showed 
less promise for future advancement and profitable service to the Navy. 
The t\-jo lower groups, however, indicated more intention to reenlitit than 
the "typical" group. 

Overall, the two lower groups tended to resemble each other, and to 
be somewhat less effective in performance than the "typicals". Where dif- 
ferences between the two lower groups were found, they tended to show the 
marginal group to be superior to the RPT group. However, most of the dif- 
ferences between the three groups were small and none of the groups had 
characteristically unacceptable perfonpance; the majority of members in 
all the groups were judged to be ser^ying the Navy adequately. 
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Two other relevant -investigations came out of the general contractual 
""^"ru^ Standlee studies. Standlee, Fattu, 

and Auble (1954b) investigated the quality of Navy technical writing by 
tabulating the frequency of words appearing in the B£ue/a<±e^' f^amjdi 

edition), Tna> ^ VouA Navy, Alt HancU, and the NavaZ Tnjuyum ^uZ- 

1 : and Fattu and Standlee (1954) applied two Flesch readability form- 
ulas to the following publications: muLiac±eX&' Manual, StmoJidsmcui 

Manual , F_^ma>z (M-iiual) , StmoAd and Cook (Manual), Comii66aJuman 
(Manual), ThAj> U VouJi Mavy, All Handi, and Navat TAxUyUng BuUeLn 
It seems unlikely that these investigations represent the first attempts 
to systematically judge the quality of technical writing ^n the Navy, but 
these are the first that appear to be available in the-official literature, 
ihe results of these investigations are predictable: Standlee et al 
showea tnat even experienced writers and teachers use too many rare words 
and exclude too many common x,7ords when relying on their experience and 
coinmon sense; Fattu and Standlee found. that standard, essential Navy pub- 
lications were, most probably, too difficult for Navy enlisted men - 
the average Flesch readability score of their sample was 61.7, which con- 
verts roughly to a nintu-grade reading level. • 

With the end of cht= Korean War, sufficient manpower again became 
available to the Navy, and in 1957 the RPT program was discontinued in 
xavor of a shift in research and administration emphasis toward problems 
connected with higher-level personnel. Subjectively. Tr.any persons in the 
na^/y (including most of the RPT graduates) felt that RPT was well worth 
the time and effort; objectively, the evaluation findings were incon- 
clusxve. 

3. Academic Remedial Training in the Pre'-ent 

h^.o ^^r'^ into literacy training appears to have 

been brought about by the Army's manpower supply problems during the Viet- 
nam conflict. During the Vietnam buildup, the Army was forced to accept 
personnel who were classified as marginal by their scores on the Armed 
Forces Qualification Test (AFQT) . Generally, these volunteers and draft- 
T /""fTu 10^h-30th pr.rcentile range on the AFQT (Project One 

Hundred Thousand, 1969). As a consequence of its manpower needs tSe 
Army indicated to the Department of Defense that a coherent program for 
procesaxng marginal personnel was needed if manpower requirements were 
to continue being met. Defense responded in October 1966, by establish- 
ing Project 100,000" which was to help meet manpower supply problems by 
spreading marginal personnel throughout all three services. The proiect 
was also intended to provide training for these men so that they would be 

P ^'^nn^n^^ '° '° ^^^^1^^^ lif^ if they chose "to do so. Under 

Project 100,000, the Navy agreed to accept up to 15% of its enlistees 
from the marginal category and allow them to volunteer for the normal 
rlTl A-=^°^dingly, the Navy established RPT units at 

or.at Lakes, Illinois, and S=in Diego, California, in February 1967 (Weeden, 
i I: immediately, the name of the program was changed to Academic 

Remedial Training (ART), and this is the name currently in use 
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Despite the expiration of Project 100,000 in 1972, the ART program 
has ^ continued in operation up to the present. Although projections have 
indicated that induction of marginal personnel will be unnecessary even 
under the current no-draft policy (e.g., Battelle et al. . 1973), ART may 
be continued because of the rising national consciousness concerning equal 
•employment opportunity and/or because of (anticipated) improvements in the 
national economy, 

A student may currently be lalected for ART if he averages between 
3.0 and 5.5 grade levels on the Vocabulary and Comprehension Sections of 
the Cates-MacGinitie Reading Test. Originally, the HavLJ ll^t series was 
once more dusted off and re-edited for the program; however, use of these 
materials was largely discontinued in 1969. Weeden (1975) describes the - 
new program that is projected for implementation in ART. The course of 
instruction may last a maximum of 8 weeks and is broken into t^.-o phases: 
a diagnostic phase and a therapeutic phase. The diagnostic phase pro- 
vides for initial assi,;i. eut of the student to one of three courses: 

!• Course Mike - A phonics course emphasizing word 

attach skills, 

2. Course Oscar - A reading course beginning at the 

3.0 grade level, 

3. Course Victor — A reading course beginning nt the 

4.0 grade level. 

The therapeutic phase consists of two mutually reinforcing areas: word 
attack and reading abilities. The word attack area concerns phonics, 
vocabulary development, and word knowledge (root words, inflections pre- 
fixes, synonyms, etc.). The reading abilities area concerns reading 
practice, comprehension skills, and study skills. 

Notably, ART is integrated only into recruit training, as were RPT 
and the Special Training Program before it. The proportion of illiterates 
xn^the United States in 1960 was estimated to be only 2,4 percent (Harmau, 
1970), and, with current screening practices, it seems unlikely that there 
is an appreciable tiumber of illiterates assigned to Navy billets. However, 
a new concern that keys on the concept of functional literacy is beginning' 
to appear. It may well be that, despite the universal attainment of lit- 
eracy by Navy personnel, some men (and women) may fail to perform because 
they do not read well enough to meet the requirements of their jobs; there 
may exist, in effect, a literacy gap. This possibility was first r.nised 
by the Fattu and Standlee report of 1954; however, two reports by Car^/er 
(1973a, 1973b) and current work by Duffy at the Navy Personnel Researcl; 
and Development Center seem to support the current concern over a liter- 
acy gap. If such a gap exists, it may become necessary to extend liter- 
acy training from the recruit coramp.ads to the fleet. The nature and 
location of programs deviscjd to. meet the lirvracy gap would represent 
very new directions in the history of literacy training in the Navy. 
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C. Final Comment 

^ From the preceding survey of literary training, it is apparant that 
problGas of literacy are not a recent discovery; they have been a con- 
cern of the Navy throughout its history. This concern has been irotivat- 
ed both by a«need for proficient job performance and by an interest in 
the general welfare of Navy personnel. In the development of literacy 
programs, the Navy has, at one time or another, been sensitive to at 
least the following six issues: 

1. Level of Literacy. Efforts have been made fo identify levels 
of literacy and to tailor instruction to the level required of a 
trainee.^ The idea of functional literacy as a consifteration in 
determining what level of literacy a trainee ought to achieve in 
order to perform a specific job has also appeared. 

2. Causp- of Literacy Problems. Adjustments in Navy programs have 
oeen made to identify speakers from non-standard English linguistic 
concr/anities, and to adjust instruction-based on the language habits 
u-ikely to be fostered by those communities. 

3. ^ Evaluation. Both narrow (Are program objectives being met?) 
and broad (Is fleet performance being enhanced?) issues have been 
considered in evaluating Navy literacy traiaing. 

4. Decoding. Issues of relating orthography to acoustic repre- 
sentation so that learners may identify information within their 
Ixnguiotic experience have been addressed. 

5. Affect. Learners' attitudes toward themselves and toward 
literacy have been taken into account. 

6. Active Participation. Most military training attempts to 
xnvolve as much "hands-on" experience as possible and this approach 
appears in Navy literacy training particularly id.th the emphasis 

on the Navy L^e. workbooks. 

On the other hand, the Navy's inter-st in literacy has been largely 
mtermitteut. From the evangelical crricern of the Chaplains to the cur- 
rent interest in ART, lack of literacy r.V.iils on the part of Navy person- 
nel has always been given full opportunity to handicap the Navy's opera- 
tional effectiveness before anything is done about it. Programs that are 
devised in response to literacy problems usually prove to be relatively 
sophisticated, as the Special Training Program and RPT curriculums demon- 
strate, but these programs seem to appear only in response to wars or 
otner national emergencies. The outlook for the current ART program can- 
not oe very bright. ART is remarkable for continuing as 'long" as it has 
a.ter the demise of Project 100,000, but it seems reasonable to antici- 
pate waning interest in ART on the part of the Navy unless problems aris- 
ing from the all-volunteer military suddenly become more severe, manpower 
supply, for whatever reason, becomes more constrained, or the national 
priorities for equal employment opportunities are sustained. 
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The primary hope for continued interest in Jlteracy training seems 
to rest on the concept of fimctional literacy in x^hich the concern is 
not so much ;d.th illiteracy as with what literacy is necessary for spe- 
cific Navy jobs (e.g., Duffy, Carter, Fletcher, & Aiken, 1975). In this 
respect it seems likely thit concerns of literacy will be modified in 
two ways. First, they will become more narrow in that finding and apply- 
ii:g technical information will be emphasized rather than more general 
skills of reading and writing. Secondly, these concerns will become more 
broad in that considerations of media, such as the organization and qual- 
ity of technical manuals, the availability and capacity of systems for 
computer-based information retrieval, and the comprehensi«ility and use- 
ability of graphic and pictorial information will be integrated wit'-, more 
conventional literacy considerations. These are fairly' novel directives 
for literacy training in the Navy, but they are also timely." It is dif- 
ficult to come away from an historical survey of literacy training with- 
out an appreciation for . the continuing increases in the amount ot infor- 
TTjation that must be available for successful performance of Navy jobs. 
Classically exponential, these increases were at first gradual, but are 
now accelerating at an impressive rate* The importance, and even urgency, 
for systematic concern x^ath literacy is hard to escape, and it is to be 
hoped that literacy training will receive contirajng support from all the 
nllitc?ry s irvices - 
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COMMENTS Oy PA^'ER BY FLETCHER 

Ri:;Le/t Qlaser 
University of Pittsburgh 

Glaser based his remarks on Fletcher's paper and on discussions hc:ld 
• at the Conference up to that time* His major point was that in both civil- 
ian and military settings, there seems to be a shift from speaking about 
general literacy and general intelligence toward the consideration of 
specific kinds of literacy and specific intelligences • He identified 
tunatiGnaZ job LU^Aacy as one type of altemati-re to the concent of 
g'ZnO^lal tU^Aacy. with ^JanCAxloiaZ job literacy, the concern is' to turn 
someone into an expert on his job - to give someone the wherewithal to 
develop expertise in his job and to develop pride in this job. He sug- 
gested that we frequently confuse functional literacy with general liter- 
acy, rather than recognizing it as the knowledge structure of job tasks 
and the interrelationships of fundamental concepts in jobs. Functional 
job literacy training should provide a knowledge network so people can 
retrieve and use job information readily, easily, and with a high degree 
of ccanectivity to other concepts so that this information can be used 
most adaptably. He mentioned the primitive theories of knowledge and con- 
ceptual networks now being thought about in cognitive phychology, and sug- 
gested they might be valuable areas for research related to functional 
literacy. In regard to gmOAaZ tutMJlCtj , Glaser stated that rather tihan 
regarding general literacy in terms of 4th, or 6th, etc, grade levels, 
ve should consider general literacy in terms of the number of opportuni- 
ties open to a person with certain literacy skills. He related the con- 
cept of aptitude to that of general literacy by pointing out that we have 
not really investigated the trainability of what we call aptitudes, and 
similarly, the grade level concept of general literacy does not focus on 
the trainability of literacy. There are a finite set of skills needed to 
get through school in the way we currently train people, and literacy 
skills make up a heavy component of this training. The more we use the 
printed page, the luore certain skills are required to get through the 
training program; this certain set of skills then becomes rarefied in what 
we call "general literacy". If we want to maximize the number of people 
getting through training, we have to analyze the basic components and pro- 
cessess making up general literacy and general intelligence and train these 
as specific^ skills. As one way to train general literacy, Glaser suggested 
first trainxng people in functional literacy; this might then provide the . 
motivauion to improve their general literacy. He also recalled the need 
to think about other ways of learning by non--literacy skills, and cautioned 
about "over-literacizing" a job; that is, making the job more "prestigef ull" 
by an unnecessary proliferation of complicated printed materials. 
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A HISTORICAL PE^^SPECTIVE OF ARMY R&D READING 
AND TECHNICAL WRITING 



Jack H. Hiller 
U.S. Army Research Institute 

Amy use of technical x^riting for training and operations has been 
and continues to be extenaive. The Army has trained at least a half mil-- 
lion illiterates since 1942 to help them to read this literature and to 
enhance their adjustment to Army life. Yet, research on reading and 
technical writing was sparse,. I will attempt to characterize this R&D 
in reading training and technical writing from the Second World War to 
the early period of U.S. combat in Vietnam. My purpose is to provide a 
backdrop against which contemporary R&D may be compared to reveal both 
progress and research gaps. 

Reading Training 

There were two major literacy (reading, writing, and arithmetic) 
programs conducted during the time frame for this review: (1) the 
Special Training Unit (STU) program of m II (Goldberg, 1951), and (2) 
the Basic Education Project at Fort Leonard Wood during 1953-4 (Goffard, 
1956). These two programs provide the focus for this review, and they 
will be discussed together as we proceed. It should be noted that the 
STU program met an urgent need and v/as not experiuiental , while the Tort 
Wood project was. 

This review is organized according to ■ a systems approach toward 
training so that the following topics will be covered: job requirements, 
training goals and objectives, instructional methods and media, selection 
and training of instructors, selection of students and achievement test- 
ing, and program evaluation. 

!• Job Requirements . Formal analyses and measurement of general Army 
reading needs were not conducted. Likewise, measurement of reading needs 
posed by specific jobs and job training was not performed. Informal, in- 
tuitive estimation of vocabulary and reading tasks provided the basis for 
training. 

2- Training Goals and Objectives . In both programs, enhanced traina- 
bility and military performance were primary goals. The STUs were furn- 
ished \<r±th relatively clear and complete learning objectives that appeared 
to be functional to Army needs. For example, the first objective required 
recognition and understanding of 46 specific Army words. Other objectives, 
were to follow directions and to find and retrieve specific information. 
The STU reading materials were carefully selected to be directly relevant 
to Army life. In contrast, the reading goals and materials of the Fort 
Wood project vzere academically oriented. 
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3. Instructional Methods and Med ia . The STUs were provided with num- 
erous training aids, such as film strips, flash cards, posters, and 
charts, as well as supplementary periodicals. The core of training was 
a specially prepared basic text and workbook that was graduated for dif- 
ficulty. All materials were Army relevant. The Fort Wood project used 
coamiercially prepared materials. 

4. • Selection and Training of Instructors . Mainly experienced, well- 
educated teachers, civilian and military, were employed by the STU. 
Super\^isors and teachers were given pre-service and in-service training. 
Carefully prepared instructor guides based on STU experience were pro- 
vided. Only civilian instructors worked at Fort Wood, 

^' Selection of Students and Achievement Testing . Students sent to 
the STU^were drawn from three categories: (1) illiterate, but able to 
learn, (2) non--English speakers with learning potential, and (3) men 
from the lowest category of learning ability. Screening was accomplished 
by a series of tests. Individuals who passed a specially developed lit- 
eracy test were given the Army General Classification Test. Of this 
group, the loxr^est performers were sent to the STU. Individuals who did 
not pass the literacy screen were given one or more non-language ability 
tests to estimate learning potential. Low scorers on the non-language 
ability tests were discharged. 

Selection at Fort V7ood was based on having a relatively low aptitude 
area score on the Army General Classification Test and a score indicating 
reading ability level below the fourth grade on the U.S. Armed Forces 
Intermediate Achievement Test. This achievement test is based on a com- 
mercial test for elementary school children and covers vocabulary, read- 
ing comprehension, and arithmetic. Both the vocabulary choices, and 
skills reflected in comprehension test items appear to have marginal 
relevance to Army needs. 

The STU attempted to place individuals according to a test specific- 
ally designed for that purpose; however, placement was intuitively decid- 
ed and definitive research was not conducted. A terminal achievement 
test, having the same form as the placement test, was used to help determ- 
ine the decision to graduate students. These tests relied on content 
validity and internal consistency to establish validity. In addition, 
they were correlated with a commercial elementary school test to support 
validity and to estimate grade equivalence. Time allowed for graduation 
from the STU varied with Army t an power needs, but overall the maximum 
allowed was about 12 weeks. graduation from the Fort Wood program re- 
quired achievement of the f ou .th-grade level within 96 hours or 16 days 
of classroom instruction. 

6. P rogram Evaluation . Fo. a brief period during the early days of 
the STU, questionnaires were sent to training units who received STU 
graduates. The training units found these questionnaires burdensome, so 
the War Department banned any further inquiries on STU post-graduation 
pel l.ormance. Evaluation of the STU therefore, must rely on criteria 
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internal to the program. Attributing effects to the STU would have been 
problematical anyivay; since its students were heterogeneous, instructional 
procedures, while carefully supervised, allowed for local innovation, and 
graduation was based on a multitude of criteria that kept changing. In 
addition, control Ss were not employed. 

Effectiveness of the various instructional programs conducted at 
Fort Wood was evaluated against external performance criteria, \^ath the 
aid of specifically selected control subjects. Higher scores were reli- 
ably achieved by the experimental subjects on both wit ten and perform- 
ance tests concerning basic military proficiency. These military tests 
were given to all trainees, including graduates of the Fort Wood liter- 
acy classes, at the eighth week of basic military training. However, 
the gain associated with the literacy training was on.ly 2-3% and was re- 
garded as practically unimportant. Supervisor ratings based on six months 
at the soldier's first duty position showed no practical effects. However, 
these negative results must be regarded as equivocal, since the graduation 
test was based on elementary school-related rather than Army-related 
achievement, and the criterion level of fourth grade may well have been 
too low to have impact. 

Summary 

Army R&D on reading training up to 1954 did not establish clear 
evidence of what reading needs exist and what effects reading training 
may have on military training and performance. 

Conclusion 

From the perspective, of a systems framework, what little research 
was conducted cannot be regarded as in any way conclusive. Specific 
reading needs x^7ere not researched. Training and job requirements were 
not specifically modified to avoid or minimize literacy demands (although 
there v/as debate on this topic in the Departments of War and Army), 
Alternative methods of instruction x^ere not extensively evaluated. The 
graduation criterion was set at fourth grade, \<fh±ch might be too low. 
In the largest training program, that of the STU, formal data concerning 
the effects of literacy training un Army performance were not collected. 
Research form the Second World War through the Korean War may be charac- 
terised as aiming at relatively low skill levels using brief training 
programs. Such brief training reflects a determined effort to provide 
'training that is cost-effective, although research to estimate costs and 
effects has been too limited to guide specification of training parameters. 
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Research on Technical Writing and Useability 

• u "^^^^ review is organized according to a system engineering approach, 
with attention given to the following selected topics: 

o Determination or specification of job functions and 
tasks perfcrned by technical writers. 

o Determination of skills and knowledge required for 
training and job performance by technical wriC&rs. 

o Development of guidelines, specifications, and -.ccept- 
ance criteria for training materials, technical docu- 
mentation, and job aids to be produced by technical 
^/riters. 

o Evaluation of training materials and job aids (e.g., 
writer's handbook) for technical writers. 

» Comparative evaluation of the ef fectivens^s jf mate- 
rials produced by writers having differen: ur.ining 
or using different Job aids. 

1. Job Analysis. I have not found any evidence of svs ^ matic research 
to determine the job functions and tasks performed by inciv ' 'i-ials who 
were assigned technical writing responsibilities. Appare.n^ >. as Rich^-rd 
r.ern ^ound by interviews conducted in the-early seventici? (reported in 
this volume), technical writing assignments were made, ou ' basis of job 
proximity, wi.h subject matter familiarity presumed to provide the neces- 
sary and sufficient expertise. Given this presumption, it follows that 
research mto job requirements would not have been pursued. 

, and Knowle dge Requirements . Although empirical research aimed 

at determining tasks pe -formed by technical writers seems not to have been 
conducted, it is clear that the communicability of Army writing was widely 
recognized to be a problem. Nevertheless, anecific skill and knoxvl-d^e 
requirements were not researched. 

3- Writing Guidelines and Acceptability Cr'-P^-i^. To alleviate problems 
caused by needlessly difficult .^-iting styles-, various writin,; guides were 
compiled; for example, Klein (1946), Stepher.son (jQ50), Milton (1953) 
Hoehn (1960), a;nd Rogers and Thorne (1965). Klein presented a guide for 
writing military correspondence, o.;d..rs, reports, manuals, and news co-v. 
He stressed that clarity and conc^-iiness were primary virtues. To hel.V 
writers achieve clarity, he showsa examples cf writing with specific 
faults and corrected versions. V.(. ./..-;o identified confusing military 
jargon, and provided brief rules of grammar and some suggestions for 
writing manuals. 

Milton prescribed the use of topic sentences, transitional techniques, 
concrete words, active verbs, and familiar vocabulary; and the avoidan e 
o.. passive verbs, parenthetical expressions, and other complex sentence 
form.9. He also advocated the Flesch notions o.. using short sentences w^" ■ h 
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personal pronouns. Stephenson reported that the Infantry School at 
-or. Bennms had adopced a" Flesch (1948) readability formula to rewrite 
Its infantry manuals. 

matiofSn '^^■-'tain logical criteria for deciding v;hen infor^ 

tnation shon.d be conveyed by a handbook insteru of by training: 

a. Ease of Communication. Consider using a handbook ^f it 
IS easy to show how to perform a given task. 

b. Difficulty to Learn. If a task is difficult to learn 
(memorize), but can be affectively supported by a hand- 
book, then skip the training in favn: of the handbook 
or support performance with a handbook. 

c. Frequency of Use. If a task is seldom performed, include 
It m the handbook rather than training. 

d. Accuracy or Safety Requirements. " Ln situations in which 
the safety of the repairman or the effectiveness and sa/eLy ' 
ot the equipment allows for no errors :n nerformance re- 
sponses should be supported by handbook content as well as 
training." (page 6). 

in. ul^f"" f described certain logical criteria for pr oducing select- 

boof'houirb'^^ "^'^ ^° ^he hand- 

c'ucta ? cost-effective by either improvias job performance or re- 
ciucmg training costs. 

ih. Included in Hoehn's report was an exemplar for L^rndbcc 1 d^sifin "n 

tovd^r^lf "^'T'"^^^^^ incl.ned to' 

tM ; Li T "^^^^fg"^ ^^^y ^'^iter could readily understand and anply 
eri use SoehV. ^^^^^^f' ^""l'^ psychologically sonhistJ ..ated writ- 

d^ct ^h ^-L°pcal criteria; for example, could t .y acc.rat..y pre- 

coPten^ learning difficulty of proposed handbook 

useSutv orrT? training? Thus, it seem, to be .hat the 

usecbility of Hoehn's logical criteria is a question. 

Rogers and Thorne actively considered factors influenci ' the d-ci- 
sxon on what to include in a troubleshooting manual, and on ho ^ ■ or'g'n- 
' t r contents. Based on their theor^tica. analysis an 

h:vin"'one "^^P-P-^^- ^n experimental test was conducted by 

having one group of technicians use the experimental manual to locse 
malfunctions in comparison to a group of technicians who uc=.d st.^^ idard 
e^nlr'' ; ^ '^^'^ ^^^^^^^^ technicians ... .n .: the 

Bas- In S """""^ P^--f°-ned "substantially" faster and more effectively. 
oar-In. ^ t ^^^^^^^h, a guide for publication agencies to use when pre- 
paring similar manuals was written (Rogers & Harris, 1964), but I h^ve 
not found evidence that it was field tested. J, ^ ^ nave 

Across the years, it is possible to see a trend from unteste-! logical 
TuToTl :i °" r ^''^ effectively (Klein, 1946) to empirl.a St- 

iSis) °" ^° P — ^ information (e.g., Rogers & T.ome, 
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A.^^ Evaluatio a of Training Materials and Job Aids for Technical 
iilii-iTJ,' ve not uncovered any instances where the utility of writ- 

ing irs!:ruction or writer's manuals was empirically evaluated. At the 
very least, the actual or attempted useage of skills developed in train- 
ing should be measured, as well as the useage of manuals on the job. If 
a writer s manual ±s never taken off the shelf, then research to design 
c Letter one might be se«u as futile. When a manual is looked at, it 

cc.er d-.ceptively s.lmple advice, such as "write short sentences using 
xaiailxar vocabulary, action verbs, personal pronouns, and few prepositions" 
wuicii .13 r.ifficult for a 'vx-iter to effect without further guidance or 
Li.axn2.n3. Furthermore, there is no solid evidence that writers who con- 
lom to prescribed rules actually produce more effective writing. 
5. Comuar ative Evaluation of Alternative Training or Job Aids . It would 
seeir: to be a point of dubious finesse to mention that comparative evalua- 
ciou;^ do not appear to have been conducted, after having cited a lack of 
evaxuatnon for singular products. However, it seems appropriate to empha- 
size tnat the ultimate objective for research on technical writing is to 
xmprcvs the readability and useability of technical documentation. Ideally 
tne products of writers who use a given experimental i^riter's manual would 
oe compared for useability with the products of writers who are left to 
utiexcoxm devices, or who use an alternative job aid. Unfortunately, such 
usaaoilxty research appears to be difficult and expensive (see Richard 
.Vem s paper in this volume) . 

OVERALL CONCLUSION 

The major impression of research on reading and technical writing 
gotten by this reviewer is that, as of the mid-sixties, there was a 
paucity of research. One might speculate on the reasons. On the one 
aand, research to improve communication has faced the spectre of theoret- 
xcal and empirical ignorance about how communication skills are learned 
anc can be taught. On the other hand, motivation to embark on potentially 
costly research programs aimed at improving training and job performance 
bv xncreasxng communication skills appears to have been relatively low 
as compared to the motivation to acquire new hardware. 
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COMMENTS ON THE PAPER BY HILLER 



Jeanne Chall 
Harvard University 

Chall made five points relating not only to Hille: ' s paper, but to 
other presentations and discussions. The first point was tha^ v^e need to 
keep in mind the fact that concerns for readability measurement and the 
witing of readable materials are not restricted only to the production 
of low level materials for beginning or low level readers. These are also 
concerns for the production of higher level materials, such as newspapers, 
textbooks, corporation reports, and so forth. Yet, the problem of read- 
ability measurement and writing readable materials may be quite different 
for providing materials for developing readers than those for highly 
Gkilled readers. She related this to the expressed concern for functional 
literacy (see Guthrie ^s comments on Burke tt's paper) in which the primers 
for the illiterates incorporated meaningful content , and contrasted this 
with the type of primers constructed to teach basic phonics skills* The 
questions of whether one or the other approach is best is open, Chall 
suggested. 

The second and third points made by Chall were that more needs to be 
known about the unskilled readers who are entering the Armed Services. 
For instance, how many are of both low IQ and unable to read, and how many 
are of normal IQ but are dyslexic — unable to read, but otherwise intel- 
ligent. For a completely adequate literacy training program, one needs 
information about the probable reasons for a person's difficulties. Chall' s 
fourth point concerned the objectives of literacy training programs within 
the Army and their practicality. She expressed her opinion that any pro- 
gram for adults which had only a fourth-grade or fifth -grade skill level 
as an objective was not preparing the person for any practical tasks. She 
wonder^ 'i why it was considered possible that one could take somebody who 
goes to school for eight to twelve years and does not learn to read, and 
put that person through a 12- (or so) week program ,ai! J teach him to do 
something (read) that a normal child would take four or five years of con- 
stant practice to achieve? Chall suggested that comparisons be made of 
adult illiterates in the military with civilian illiterates and \<r±th chil- 
dren who are learning to read at different ages. From such comparironF. , 
military' researchers might gain useful information about the asp.-cts of' 
learning to read that are universal to all and about those .specific to 
adults^ and further, to adults in the military, ^^here theie is commonality, 
much can be learned from the extensive research that already exists, bn.sed 
on children. 

The fifth and final point made by Chall was that there needs to be 
materials of a suitable reading level around for newly trained literates 
to practice reading. If Army men are taught to read at the fourth-grade 
level, and there are no Army materials written at that level, then there 
is very little opportunity for further practice and skills may dwindle to 
pretraining levels. Army and other military literacy training systems 
must be designed to promote the continued growth of newly acquired, fragile 
reading skills. 
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A REVIEW OF RESEARCH AND DEVELOPMENT 
ON LITERACY AND TECHNICAL WRITING IN THE AIR FORCE 



James R. Burkett 
Air Force Human Resources Laboratory 

INTRODUCTION 

Problems relating to reading, literacy, and technical writing have 
persisted in; the Air Force for many years. Because of the impact of these 
problems on training, job performance, and operating costs, as well as the 
considerable complexity of the literacy area from both a men and a materials 
standpoint, numerous R&D efforts have been carried out addressing various 
aspects of literacy and technical writing problems in the Air Force, The 
purpose of this paper is to summarize the major research directions the 
Air Force has pursued in these areas over the years in order to provide 
some historical perspective . on what has been accomplished and what has led 
up to the present RSD programs on literacy and technical materials. 

3efore reviewing the various Air Force research efforts concerned 
with the literacy area, some perspectives on the nature of the problems 
that have generated the research are in order. Simply stated, both lit- 
eracy and technical writing problems arise from a situation where there 
is a mismatch (or discrepancy) between the reading demands of job or train- 
ing materials and the reading abilities of personnel who must use those 
materials. This mismatch between the literacy skills of men. and the read- 
ability or useability of their materials is often referred to as a *'lit- 
eracy gap'\ The existence of a literacy gap interferes with the success- 
ful transfer of information from written materials to the personnel who 
must use thaL information for the purpose of training or job performance. 
The Air Force has alx^yays had some numbers of lower or limited ability 
personnel and has also experienced an ever increasing technical complex- 
ity of job demands and materials, even in high ability career areas. 
Accordingly, a primary goal of Air Force researchers, personnel planners, 
and training managers over the years has been 'to identify and reduce mis- 
matches between men and materials as they have occurred, and a variety of 
techniques and approaches toward that goal have been developed and employed 
with varying success. The important consideration here is that the prob- 
lem has never been found to be the fault of the men or the materials alone. 
Both have contributed to the magnitude of the recurring literacy problems 
faced by the Air Force, and research has had to deal with both to some 
degree to develop t e tools and techniques to address the problems that 
have persisted through the years. 

Thus, the goals of improving both literacy skills and technical mate- 
rials have proven to be complex and difficult problems for the Air Force, 
requiring recurring, if not altogether ongoing, research and development 
emphasis over the years, an emphasis which continues today in the current 
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progran of the Air Force Huiaan Resources Laboratory. The work accomp- 
lished throughout the history of Air Force research on the problems of 
literacy and technical writing spans a wide variety of topic areas and 
approaches. Major areas of emphasis have included efforts on readability, 
nechnical writing and formatting, procedures for dealing with lower ability 
personnel, job performance aids, useability of technical materials, simpli- 
fication of naterinls, lit-racy skills and literacy training, determining 
reading requirements, and measuring textual comprehensibility • \^ile the 
R&D efforns accomplished to date have provided a useful technology base 
and a variety of useful techniques and applications in these areas, the 
v/ork continues. This is a reflection not of the irrelevance or lack of 
success of earlier efforts, but of the scope and complexity of the problem 
area and of the ever changing nature of Air Force jobs and manpower. 

Vve shall now turn to a review of Air Force R&D on literacy and tech- 
nical writing. First, early efforts will be described. Next," the work 
that emerged from programs of the Air Force Personnel and Training Research 
Center (AFPTRC) from the middle through the late fifties will be reviewed, 
along vich the research relating to lower ability airmen that was contin-* 
ued by subsequent organizations through the middle sixties. Then, the 
I'xstory of the Air Force program on Job Performance Aids will be briefly 
described. Other research of interest that has occurred outside the main 
Air Force personnel, training, and human resources R&D channels will then 
be noted. Finally, recent research on literacy and readability will be 
nientioned to complete the historical perspective of R&D accomplished to 
the present time. Throughout the review, both Air Force in-house and 
Air Force sponsored contract research are included, if the work appeared 
relevant to the topic areas:. Uliile it is recognized that much of the 
Air Force R&D on literacy and technical writing has been accomplished by 
contractors, no attempt has been made in this review to separate the con- 
tract efforts from the in-house work. 

EARLY EFFORTS ON READABILITY, 
TECHNICAL WRITING, AND LITERACY 

In the early fifties, very soon after the* establishment of the Air 
Force as a separate service department, interest in research on readabil- 
ity of job and training materials c.nd research on utilization and training 
of low ability personnel emerged as important concerns. The Air Force was 
becoming aware of the fact that much of its job and training materials 
were very difficult to read, and that the increasing influx of lower abil- 
ity airmen coming in as part of the Korean Conflict manpower buildup was 
compounding the problem. England (1950) used the Flesch (1948) readabil- 
ity formula to check Air Force publications and found them to be unneces- 
sarily difficult. He advocated a cai^paign for use of ^'plain talk^' and 
simplification of materials in accordance with Flesch 's reading ease 
guidelines. A year later (England, 1951) he reported cin analysis of the 
effects of a one-year application of his "plain talk" principles to USAF 
Air Materiel Command publications. Again, using the Flesch formula to 
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check the raadability of his, simplified publications , England found sub- 
stantial gains in reading ease and comprehension among users of the sin- • 
plified materials. He advocated their expanded use, but it would appear 
taat his campaign for "plain talk" had only a limited and somewhat tran- 
sient impact on the problem of too difficult job materials. 

Training Personnel were also becoming interested in the potential 
application of popular readability metrics for improving their instruc- 
tional materials. Hegg and Weaver (1952) evaluated the reading diffi- 
culty of training materials at Vance AFB using the Flesch formula and 
found that the training literature sampled was written at a level far 
above the reading ability of the middle two-thirds of the students who 
had to use the materials. Vfhile they offered no suggestions for specific 
ways to simplify the trailing materials, they did indicate that instead 
of writing to a 12th- or nigher grade level, writing to nbout a 9th-grade 
level would better match the reading capabilities of nost of the trainees. 
Peerscn (1953) reported on a readability improvement arogran for locally 
produced training materials at Kee.iler AFB. This tim2, the Dale-Chall 
(1948) readability approach was used, and again the ilea was to reduce 
the difficulty of the training literature to a level n ore appT-opriate for 
the students. Milton (1953) suggested that technical writer- should take 
the initiative in improving the readability of their material^; bv using 
a variety of available techniques (such as those advocated bv Fl-rch) for 
makins their writing less complex, more direct, and easier to read. The 
emerging interest in measuring and improving readability of Air Force 
writing was officially recognized in 1956 with the pubJ ication of a Guide 
CO/I Ao'c Fo4C£ WMing (AM 11-3). This manual gave instructions for cal- 
culating the Fog Count readability measure developed by HcElroy around 
1953 and recommended the application of a variety of effective writing' 
principles, including those of Flesch and others, by Air Force personnel 
I'Thile this guide was a useful step, the mere issuance of such a manual 
could not resolve the overall literacy and technical '.writing problem. 
For one thing, an increasing influx of lower ability personnel in the 
early fifties had resulted in training problems of sufficient magnitude 
to make literacy a major concern. 

The Air Force had always had some low ability personnel (mental Cat- 
egory IV on the AFQT) and some effort had been directed toward reducing 
their training problems through special programs to delete difficult read- 
ing materials from their curriculum and to provide extended course lengths 
CO give them more time in training (Horton, 1951). However, by 1952 the 
problem had become more acute, and the Air Force undertook a comprehensive 
series or studies of special training programs for limited aptitude rirmen 
xv-ith marginal literacy skills. The first study in this series (Gragg 
Kieselbach, Murphy, Peckham, & Heller, 1955) attempted to determine the 
effectiveness of covering the same material then in the standard 8-week 
basic training course (which included language arts) at a slower nace in 
an expanded 14-week course using matched groups of marginal airmen. The 
two training programs differed only in the length of training time pro- 
videa. The performance of the two groups was compared at the end of their 
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programs on various measures,- and no appreciable, differences between them 
were found- There was no difference iu language arts (as measured by the 
California Achievement Test) even though a major portion of the expanded 
time in the 14-week course had been Cavoled to that area. It was con- 
cluded that little gain could be expictt^d for low ability airmen from just 
extending the training period and covering the standard material at a 
slower rate. A more extensive study was then undertaken which incorpor- 
ated a larger sample of marginal ainien, two new experimental curriculums, 
and extensive field follow-up comparisons between the experimental low 
ability groups and control groups of higher ability men. 

The new study was known as "The Air Force 'Project 1000' Study". 
The subjects were 1000 limited aptitude airmen. One-half of them received 
basic training in experimental six-week curriculum and the other half 
in an experimental twel■^^e-week curriculum. The two sample groups were 
matched on age, race, marital status ^ and prior education. The two train- 
ing programs differed primarily iu the inclusion of 45 hours of language 
arts, and 45 hours of mathematics training in the longer course, and in 
the expanding of instruction in military fundamentals from 18 hours in 
the six-week course to 35 hours for the twelve-week course. Mastropaolo, 
Carp, Erdmann, and Schmid (1954) reported the relative effectiveness of 
the two training; programs on a variety of criteria obtained at the end of 
training and after six weeks of initial duty in the field. At the con- 
clusion of training, the twelve-week group showed a small margin in 
arithmetic skills over the six-week group, but none in reading skills or 
language arts. When compared six weeks after assignment, no differences 
were found between the two groups on measures of job proficiency, atti- 
tude, or adjustment. A control group of "normal ability" airmen who had 
undergone the standard training course that were working alongside the 
low ability men were found to be superior on job performance ratings and 
achievement test scores, although somewhat inferior to the low ability 
groups on attitude and adjustment variables- It was concluded that for 
practical purposes, the longer course was not superior to the shorter one 
for the low aptitude aimcu. 

An eight-month-after-assignment field follow-up study was also con- 
ducted (Mastropaolo, Carp, & Erdmann, 1954) and the findings were generally 
consistent with those of the first study showing that the two training pro- 
grams had produced equivalent results in tc lths of on-the-job performance, 
attitudes and adjustment. Again, d control group was found to be higher 
in job proficiency ratings and achievement test scores than the Project 
1000 airmen. Thus, the results of the eight-month follow-up indicated 
that the low aptitude men had not benefited from expanding their training 
in terms of their job proficiency, retainability , or conformity to other 
Air Force standards of performance and discipline. It also appeared that ' 
those marginally literate men who had received additional training in 
language arts and mather.atics during the longer course were not appreciably 
different in their skills, knowledges, and adjustment to the Air Force than 
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r.ho-se marginal airmen who had not had such additional Instruction. The 
value of including language arts instruction during basic training 
appeared questionable. ^ 

n.cc ^^^^^y/"^ Smith (1955) attempted an evaluation of the effective- 
ness of Che language arts aspect of the basic training program for low 
beii' "T."" (Category IV on the AFQT) to determined? the procedures 

^avin.'thf f^^T?'""- '""'"'"^ "^^'^^^S' ^^^^-S' -P-lli-g were 

uZTr: ^ desired impact. Three groups of 140 marginal aimen each 

guLT^'^sT^ith ^-^---'g P-g-ams: (1) 45 hours in lan- 

guage arts with military subject matter as the vehicle, (2) training in 

and ""'''^^ ^° ^-i^i-g in language a?ts 

and (3) the regular non-remedial training. Scores of the groups on ^e^d- 
xng comprehension and military subject matter tests at the and of train- 
ing indicated tnat the language arts program was effective in improvSg 
there'wasir"' °' I"!"'" ^^terl.l. However, it was also noted that 
Ind that tL n variability in the degree of improvement across trainees, 
S evon P^^vious studies had shown basically negative results with 

Uteracv tra^-n^f "iH r^K^ concluded that remedial 

t.nZ """^^ expected to show some benefit for marginal 

groups with a fairly wide range of abilif -, and that such programs ^i^ht 

de'f^Hirfrorth"' as criteria for separating tho.e men witE a^L^ta! ' 
aeticit from those with an educational deficit. 

RESEARCH ON READABILITY, TECHNICAL MATERIALS 

AND LITERACY SKILLS FROM THE MID-FIFTIES THROUGH THE MID-SIXTIES 

h,l ^^^t'u^^^.^^'' ^"''''^ Personnel and Training Research Center (AFPTRC) 

had been established, and this organisation had a large-scale and diverse 
RuD program well underway. As will be noted later, the extensive Air Force 
R&D program on Job Performance Aids (JPAs) originated within AtTTRC during 
was carried' r T""' °t ''""'^^ different from those specific to JPAs ' 
of rit ""^""^ ^1^^^ ^"'i his associates as a result 

°his wo'r''^'t"^ improving Air Force technical materials. 
tol loS . '^loi'o''-^ ''^''^ ^"'"^ °f experiments and studies published 

from 1954 to 1959 investigating the relationship of a wide variety of com- 

i^ fn""'. readability variables to the comprehension lea".- 

ing, and retention of technical materials. - 

nqSA?'' initial report in this series, -Klare, Mabry, and Gustafsou 

C1954) present .he results of a number of experiments on the effects of 
unSr ^^^'^f ^l^^y abstraction, use of personal words and 

underlining key words and phrases on the effectiveness of written technical 
dSayeT^tL't' ^^^-^-eness was assessed by measuring inm^edJ^^e and 

-trlTr ? f .^^ °^ material, reading speed, interest value, and 

-tlTI ^-t / °- attitudes toward the material. Incr^asSg 

.eadabilxty (as measured by the Dale-Chall, 1948, and Flesch, 19^8, fonn- 
ulas) was found to produce increased immediate retention ^nd reading sp^ed 

St SsT.r r'^\'^'^ ^-^^^^'^ interesting by thTsub 

incrP^.p personal rather than impersonal words did not appear to 

increase che interest or acceptability of the material. 
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Klare, Mabry, and Gustafson (1955a) described their attempt to relate 
patterning (underlining) of technical materials to immediate retention, 
amount read in a given period of time, and acceptability of the material. 
Underlining important words resulted in more liries read and in higher re- 
tention than either underlining words which appeared on tests or no under- 
lining, but^he^if ference was not significants. It was further noted that 
patterning as used~~in this experiment appeared to have little impact on 
either reading speed or acceptability. Klare, Mabry, and Gustafson (1955b) 
further explored the question of holding content and style difficulty of 
technical material constant to determine if increasing hiiman interest in 
Such material would alter immediate retention, acceptability, and reading 
speed. Two levels of human interest were used, varying in both percentage 
of "personal words" and percentage of "personal sentences" • The high level 
scored 28 ("interesting") on the Flesch formula, and the low level scored 
zero ("dull"). Results indicated that a high level of human interest of 
the technical xvrriting produced no significant difference in retention and 
v/as consistently judged less acceptable. However, it did show a slight 
tendency to produce a greater amount read in a given time as compared to 
the low human interest level material, 

Klare, Mabry, and Gustafson (1955c) further elaborated their findings 
on attempts to relate style difficulty to immediate retention, reading 
speed, and acceptability of technical training material. An easier style 
of writing (as measured by the standard formulas) was found to result in 
greater and more complete immediate retention, a greater amount read in a 
given time, and more acceptable reading material. This study also indi- 
cated a greater importance of content than style in determining how well 
material will be accepted by trainees. In addition, it showed a high re- 
lationship between judgements of material as easier to read and more 
pleasant to read. Another study in this series (Klare, Gustafson, Mabry, 
& Shuford, 1955) was conducted to explore the relationships of irunediate 
retention test scores covering a technical training passage, selected 
"career preferences" in the form of expressed interests,- and certain air- 
men aptitude indices. No significant relationship was found between car- 
eer preferences and retention test scores, but a highly significant posi- 
tive relationship was clearly indicated between aptitude scores and 
r^etention. 

Building on the earlier studies in the series, Klare, Shuford, and 
Nichols (1957) examined the relationship between style difficulty of tech- 
nical passages, practice in the form of additional readings, and aptitude 
to both reading efficiency and retention. Results indicated that an easy 
style produced significantly higher reading efficiency and retention, and 
that additional practice increased words read per second, recall, and word 
recognition. Higher ability airmen read more efficiently and with better 
retention than lower ability men. In another study in the series, Klare, 
L^ichols, and Shuford (1957) assessed the effects of three types of typo- 
graphic arrangements and long and short "thought units" on reading effi- 
ciency, acceptability, and immediate retention of technical material. 
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Results suggested that a square-span and a spaced-unit arrangement might 
possess soTue advantages over 'a standard sentence arrangement. The new 
arrangements vere found to be less acceptable to the trainees than the 
standard /sentence format, but this feeling was less \<ihen the "thought 
units" in the new arrangements were small rather than large. 

Klare, Shuford, and Nichols (1958) reported a study of the relation 
of format organization to learning technical material. The use of heads 
and side heads in a text x^as tried as a learning aid, and the effect of 
change^' in levels of organization of the material on its acceptability 
to trainees was also assessed. No significant increase in reading speed 
was foUnd for higher levels of organization over lower levels, but train- 
ees significantly favored the higher levels in terms of acceptability of 
material. Scores on an immediate retention test also appeared t'o favor 
higher levels of organization. Thus, this study did demonstrate an in- 
crease in reader accepteince for more highly organized as compared with 
less organized material. The question of whether the addition of head- 
ings^ increases immediate retention and comprehension was unresolved , but 
this study provided at least a qualified positive answer. In a study re- 
lated to the previously described series, Stolurow and Newman (1959) 
attempted to isolate patterns of relationships among previously identi- 
fied stylistic features of printed material in order to obtain a smaller 
number of more fundamental variables for research use. Gray and Leary's 
(3/935) intercorrelation matrix was reduced from 44 to 23 variables and 
f Actor analyzed^ After rotation, two factors tentatively described as 
e/asy versus dif:Scult words and difficult versus easy sentences were found 
to account for approximately 50% of the variance, and the factor analysis 
provided useful new insight into the relative importance of these and 
/other factors for readability measurement. 

Finally, vze shall consider an AFPTRC study not related to- the previous 
series, in which Rubenstein and Abom (1958) examined the interrelationship 
between learning, word-prediction, and readability. Correlations computed 
between amount learned, the number of correct predictions per word, and 
Flesch and Dale-Chall readability- scores indicated the following relation- 
ships: Learning, prediction of words, and readability were closely inter- 
related; prediction and readability correlated about equally well with 
learning; the Dale-Chall formula correlated more closely ^d.th learning 
than the Flesch formula; and, despite the differences between the two read- 
ability formulas, they were found to correlate very highly with each other, 
which supported the findings of other researchers suggesting that the two 
formulas measured substantially the same things. 

All of these studies of ; various aspects of readability and technical 
materials through the late fifties contributed to the beginnings of an 
Air Force technology base in the technical writing area which later efforts 
V70uld build upon. Also during this period and into the mid-sixties, per- 
sonnel researchers x^ere actively engaged in studies and analyses of ability 
and educational requirements, personnel skills- and* chara rteristics , and 
other matters relating to low literacy and low aptitude airmen which the 
Air Force still had in limited numbers. 
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HcReynolds (1958) noted -that the quality of career enlisted person- 
nel was up during the 1957-58 period, and that the rapidly increasing 
coir.plexity of Air Force weapon systems would require additional high 
aptitude career personnel in coining years. Airmen in later enlistments 
ware found to be typically higher in aptitude than first term airmen, 
end the need for continuing numbers of high ability personnel to , man 
the highly technical career fields was noted. Flyer (1959) found that 
low educational level was the best single predictor of unsuitability dis- 
charge, and Judy (1960) examined educational requirements for various 
specialties and f,ound that high school graduation was correspondingly 
the best single, predictor of success in Air Force technical schools. 
Gordon and Bottenberg (1962) subsequently confirmed that of the variety 
of individual predictors of Air Force success, the amount of education 
attained was the most valid, further justifying the Air Force in limit- 
ing recruitment to high school graduates. 

Lecznar (1962) traced the aptitude data on Air Force enlistees from 
1956 through 1961, and reported that there had been a consistent upward 
trend in the overall aptitude level of enlisted accessions during that 
period. The data suggested that the selective recruiting program insti- 
tuted in 1958 had in fact upgraded the aptitude quality of Air Force en- 
listees as compared with the previous enlistment base. It was further 
noted that, to the extent possible, the Air Force stood to gain qualita- 
tively by enlisting only high school graduates. However even during this 
period, it had not been possible to completely exclude all low aptitude 
enlistees ; and duirng 1961 alone over 2,000 entered the Air Force. 

Gordon and Flyer (1962) examined the performance characteristics of 
a group of over 11,000 low aptitude aiinnen who entered the Air Force dur- 
ing the first six months of 1956 and who had either successfully completed 
their four-year enlistment or had been discharged for unsuitability or non- 
advancement. The objective of the study was to devise a brief screening 
battery that could be used to predict the success of such low ability re- 
cruits in their first enlistment, since it was recognized that the Air 
Force x^ould continue to be faced with the periodic necessity of lowering 
its enlistment standards to meet its ever changing manpower quotas. It 
was found that a brief composite of aptitude tests and preservice educa- 
tional level differentiated the successes from the failures quite well. 
It was also pointed out that the data supported a general policy of cur- 
tailing enlistment of low aptitude men, since almost 50% of them could not 
be expected to perform effectively over a four-year enlistment (a failure 
rate about twice that experienced for higher aptitude enlistees)* 

Later, McRejmolds. (1964) conducted a similar study of failure in basic 
training and found the rate of eliiTiination for high school graduates to be 
only about one— fourth of that for nongraduates. Again, it was suggested 
that some additional screening should be employed to predict the potential 
success of nongraduates when Air Force quotas could not be met by enlist- 
ment of recruits who had finished high school. Vitola, Valentine, and 
Tupes (1967) conducted a new analysis summarizing the trends in aptitude 
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and educacioual data for Air Force enlistees, this time covering the 
period fron 1962 through 1965. They found an overall upward trend in 
aptitude, and educational levels of airmen accessions during this period, 
siinilar to that previously reported in Lecznar's analysis of the 1956 
through 1961 accessions data. There werealsb expected indications of 
positive relationships between education level and measured aptitude, and 
between rec^ion of enlistment and aptitude. High school nongraduates were 
^Sain found to have lower average aptitudes than enlistees who had com- 
pleted high school or high school plus some college. 

From these personnel studies carried out through the mid-'sixties , it 
IS apparent that, the Air Force was basically following a policy of selec-« 
tively recruiting its way out of the literacy problem by taking in only 
limited numbers of low ability personnel, and stressing the sustained 
enlistment of high school graduates. However, as will be mentioned later 
xn this historical summary, the situation was about to change ^cLth the 
advent of Project 100,000 in 1966, and this change would bring about a 
rer.ev;ed interest in literacy and technical writing problems within the 
Air Force research program on Job Performance Aids which has spanned the 
period from the middle 19503 to the present time, and which has comprised 
a major portion of the Air Force effort in the technical ^^iting and use- 
abilicy of materials areas over the years. 

RESEARCH & DEVELOPMENT ON JOB PERFORi^lANCE AIDS & RELATED TOPICS 

The Air Force R&D program on Job Performance Aids (JPAs) originated 
in the middle fifties within AFPTRC, and it has been actively pursued 
ever since by a number of subsequent research organizations. Even with 
uhe various fluctuations in funding and available resources . that have 
occurred over the years, the total scope of the program has been so ex- 
tensive that the area can only be . highlighted here. However., major as- 
pects of the JFA effort have dealt with matters directly relevant to Air 
Force literacy and technical writing problems, and much of the initial 
interest in informational JPAs grew out of early concerns in areas related 
to the redesign of manuals, checklists, and other written job materials to 
improve their useability and intelligibility. You will recall that by the 
Biidrfif ties, there was an increasing Air Force awareness that many train- 
ing and job materials were poorly designed and unnecessarily difficult- 
This problem was of particular concern in the maintenance area, where tech- 
nical orders and procedural manuals were critical to successful job per- 
formance and proper repair and troubleshooting of complex equipment. The 
JPA concept was conceived as a potential solution to the problem. Essen- 
tially, JPAs Xv-ere defined as special items or devices designed to enhance 
or guide on-the-job performance. These included a variety of instructional 
manuals, proceduralized checklists, detailed diagrams, or other information 
storage devices and materials which could be provided to the technician to 
maximize his capability to retrieve and use needed job information in the 
exact sequence required. 
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Berkshire (1954) developed and evaluated an early set of simplified 
troubleshooting materials for radar mechanics^ These materials included 
a set of color-coded schematics of the equipment, along with detailed 
x/ritten troubleshooting instructions. Tryouts of the materials demon-- 
strated that use of the troubleshooting aid produced considerable time" 
savings and grciatly reduced errors made by both experienced and inexperi- 
enced technicians. Subsequent work resulted in a series of reports pro- 
vidin!;^ guidelines for the improved prep/iratiou of maintenance information, 
job instructions 5 checklists, drawings, and diagrams (Folley & Altman, 
1956) and maintenance manuals and handbooks (Miller, 1956 and Ne^>7man, 
1957a, b). 

By 1958,. a substantial amount of x-york had been accomplished, includ- 
ing development of improved task analysis techniques for designing per- 
formance aids (Wulff & NevTman, 1956; Gunn, 1956; and Emeson & Wulff, 1957), 
research on the design of troubleshooting guides (Hoehn & Salts, 1956; 
Hoehn & Wardell, 1957; and Hoehn St Aukes, 1958), and a study of audio JFAs 
(Lumsdaine & Hoehn, 1958), Hoehn and Lumsdaine (1958) summarized the work 
done by ATPTRC on job aids up to 1958, including the concepts that had 
been developed concerning the role of handbooks, films, and other forms 
of aids in enhancing job performance and the techniques developed for 
sy.qtematically planning, preparing, and testing integrated job aids, 
tv. ainihg materials, and troubleshooting guides. 

VJith the closing dox^Ti of AFPTRC early in 1958, responsibility for 
the JPA work \^a.s moved to the training and engineering psychology research 
activities at Wright-Patterson AFB, and the continuity of the program x^as 
maintained in spite of a variety of organizational changes that occurred 
there through the late fifties. Effectiveness and useability of check- 
lists (Rees & Kama, 1959) and technical manuals (Ross, D.A. , 1959) were 
further evaluated, and research on improving the design and formatting of 
job guides and maintenance checklists was continued (Rees ^ Copeland, 1959 
and Rees, 1959). By the early sixties a systematic JPA R&D program x^as 
aj/,ain underway. 

Folley (1961a) described the potential contribution of performance 
aids to improving system effectiveness and outlined a variety of research 
problems associated with integrating the design of such aids into the sys- 
tem development process. Folley and Hunger (1961) reviewed the literature 
on the design of informational JPAs and noted that most of the prior work 
had involved development and tryout of various sample aids, as well as 
research on design and use of procedural aids for improving maintenance 
troubleshooting performance. Folley (1961b) presented a systematic pro- 
cedure for developing performance aids which incorporated four key steps: 
identification of the task elements the aid should address, determination 
of required functional characteristics of the aid for each task element, 
specification of the physical design of the aid to perform the required 
functions J and evaluation for modifying or updating the aid as necessary, 
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The procedure was successfully applied to developmenc of a set of aids 
for a missile system (Folley ,&• Shettel, 1962) and was refined by further 
work through the mid-sixties. 

During this perioda^^ on through the late sixties, research on the 
development yand effectiveness of JPAs was pursued in order to expand the 
.data base on their potential application and payoff to the Air Force. 
Elliott (1965) studied the effects of varying the format and level of 
detail of performance aids on a troubleshooting task and found that per- 
formance was generally better with a block diagram format than a list 
structure format, and that speed of task accomplishment was increased by 
use of. a lower level of detail. Other studies demonstrated that p^oced- 
uralized troubleshooting could produce acceptable or better performance 
on complex maintenance tasks than standard approaches, while also per- 
T^Q^7^° ^"""^ ^^'^ training savings (Elliott, 1967). Folley and Elliott 
a967) conducted a field survey of electronic maintenance technical data 
and concluded that the performance aid requirements of electronic tech- 
niques were not being adequately met, even though a variety of proven 
pertormance aid techniques and concepts were then available. A series 
of studies undertaken to provide an experimental comparison between pro- 
ceduralized and conventional electronic troubleshooting (Elliott & Joyce 
1968) further demonstrated that the use of the proceduralized perfomance 
aid approach produced substantial savings in training time with no appar- 
ent decrement in task performance. 

By the 1970s, the JPA approach had been sufficiently demonstrated 
that increased effort could be devoted to its dissemination and applica- 
tion, and various guidelines and handbooks on design and use of fully 
proceduralized JPAs were produced. These included the following: spe- 
cifications for preparation of fully proceduralized JPAs (Chenzoff , Mal- 
lory, & Joyce, 1971; Folley, Joyce, Mallory, & Thomas, 1971a; and Joyce 
Chenzoff, Mulligan., & Mallory, 1973a); handbooks for development of JPAs 
(Folley, Joyce, Mallory, & xhomas , 1971b; Foley, 1972; and Joyce, Chenz- 
off, Mulligan, & Mallory, 1973b); and handbooks for managing the procure- 
ment of JPAs in accordance with the specifications (Joyce, Folley, & 
Elliott, 1971 and Joyce, Chenzoff, Mulligan, & Mallory, 1973c). The "fPA 
R&D program has continued through the present time and recent studies^have 
attempted to develop improved JPA formats (Joyce & Chenzoff, 1974) and to 
demonstrate expanded applications of the JPA concept (Mullen & Joyce, 
1974). Current efforts involve the development and evaluation of inte- 
fmf^^^r^^ °^ ^"-^^^ proceduralized JPAs for entire systems, such as the 
UHUi Helicopter and the Cl4l aircraft (Shriver & Foley 1975- Foley 
1975a, b; and Shriver, 1975). ' ' . ' 

The techniques and concepts that have emerged from the JPA R&D pro- i 
gram have had a positive impact on literacy and technical writing prob- 1 
l^ms in the Air Force, and present programs tc improve Technical Orders I 
(TOs) and maintenance publications are applying principles growing out of i 
JPA research. Also, the JPA approach appears to have produced c. viable ' 
^i^^r'^l^^^ the literacy demands of manuals and job materials 

which should see increasing application in the future. j 
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OTHER RESEARCH ON READING AND READABILITY 

A number of Air Force studies of reading and readability have occurred 
over the years that were conducted outside the ongoing research pro,gr^s ' 
of the prinary personnel, training, and huaan resources R&D organizations. 
One of tha most significant among these was Taylor's (1957) development 
and validation of the "cloze" technique for measuring readabiUty, which 
nas seen widespread application in a variety of settings both within and 
outside the Air Force. Also notable has been the work of Davis at the 
Air Force Institute of Technology (AFIT) on the influence of a variety of 
textual, content, format, and personal variables on the effectiveness of 
technical communication via written materials (Davis, 1965, 1966, 1974a, 
D, c) and an AFIT thesis analyzing the relationship between readability 
of Air Force procedural manuals, literacy gaps, and frequency of discrep- 
ancies involving actual noncompliance with procedures on the job (Johnson, 
Relova, & Stafford, 1972). 

RECENT RESEARCH ON LITERACY AND READABILITY 

From the mid-sixties to the present time, the Air Force has maintained 
a very active R&D program in the. literacy and readability area that has 
been carried on by -both training and personnel researchers. A bit of back- 
ground may help to place this recent research . into some perspective. You 
will recall that from the .mid-fifties through the mid-si:j;ties the Air 
Force had been pursuing a selective enlistment program which had allowed 
recruitme..t on the basis of both aptitude and educational level and which 
had kept the overall quality of accessions fairly high, l^ile limited 
numbers of low ability men had been enlisted throughout this period, they • 
had not entered in sufficient numbers to present a major literacy problem. 
However, in 1966 the situation was radically altered with the initiation 
of Project 100,000 by the Department of Defense, and this brought increas- 
ing numbers of low ability personnel into the Air Force and other Services. " 
The objectives of the program required that certain quotas of marginal 
personnel be accepted and further specified that these personnel be given 
the opportunity to enter a wide variety of career areas and job specialties 
that had not previously been open to them. Thus, Project 100,000 resulted 
in a rapid increase in input of men with marginal literacy and aptitude 
skills, and the Air Force was faced with the necessity of considering a 
variety of measures to adjust training and job demands to accommodate them. 
As a result, increased interest in research on literacy and readability 
problems emerged as a major concern. 

Two problems became inimediately apparent- First, there was little 
data on the reading abilities of Air Force personnel across career fields, 
and large scale reading testing on a long-term' basis would prove costly 
for making assessments of the scope of the Air' Force's emerging literacy 
problem. Purthermore, it would be difficult to do much about the problem 
of adjusting reading demands \Nrithout some indication of the reading abil- 
ities of personnel, which could be used for recommending appropriate 
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readafaxlity levels for mater-ials. Second, it would be similarlx diffi- 
cult .to check the readability of the large amounts of written training 
and job materials produced by the Air Force without some automated al- 
ternative to the hand-calculated metrics that had been in use since the 
early sixties. Research had been initiated prior to the advent of . 
Project 100,000 that provided timely answers to these problems. 

Madden and Tupes (1966) had already developed conversion tables for 
estanating readxng achievement from the AQE General Aptitude Index in 
ter^s of reading grade level as measured by the California Achievement 
-est and scaled score as measured by the Davis Reading Test. They had 
also provided distributions of estimated reading grade level for non- 
prxor-service airmen and for airmen in 29 Air Force career fields. The 
conversion procedure for estimating reading ability has been in use ever 
since, and the career field data served as the standard source of target 
levels for readability of Air Force materials for almost a decade. 

Also by 1966, an Automated Readability Index (ARI) had been devel- 
oped (Smith & Senter, 1967 and Smith & Kincaid, 1970). The ARI provided 
an easy, automated method of collecting the data required to estimate 
readability^ of textual material by mechanical tabulation of-word and sen- 
tence length as the material was typed on a standard typewriter. Insert- 
ing these data into a simple formula provided an index of the reading 
difficulty of the material. The feasibility of using the ARI to assess 
the comprehensibility of Air Force technical orders was subsequently 
demonstrated (Kincaid, Yasutake, & Geiselhart, 1967) and the ARI has been 
periodically used for that purpose and various other applications up to 
the present time. rr f 

■ In 1968, the Air Force Human Resources Laboratory (AFHRL) was estab- 
lished, and the Technical Training Division of the Laboratory was given 
prime responsibility for Air Force reading, literacy, and readability 
research. The Division has been actively pursuing work in these areas 
since its establishment in 1969. From its inception, this R&D program 
has been planned and carried out with a long-term view toward research- 
ing the literacy problem systematically; that -is, to have the research 
address major aspects of the problem from both a men and a materials 
standpoint. 

Initial effort was placed on researching techniques for improving 
materials and readability. Sellman (1970, 1972) evaluated a modified 
career development course (CDC> format to determine if reducing the read- 
lag aixriculty of the material, including more illustrations, and provid- 
ing audio supplementation of the text would improve the CDC as a training 
device for lower ability airmen. The simplified materials with audio 
supplementation produced signif icanlty better learning scores for high 
middle and low aptitude groups, with the greatest gains obtained by the 
lower ability trainees. At about this same time. Huff and Smith (1970) 
compared tne reliability of the Fog Count with that of the Automated 
Readability Index (ARI) and established ARI baseline data on a sample of 
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Air Force CDCs. The Fog Count, then the accepted Air Force readabil^-tv 
on^r^^q IT^'V^ ^^""^ unreliable with coefficients ranging from 

^^li^bility coefficients for the AM, on thither 
an T to .99r, and it was concluded that the ARI offered 

an extremely reliable tool for assessing the relative readability of 
^M^r . ^ ! was also pointed out Chat the ease and rapidity with 

of S/!?T ^^^^^'^ ^^^^ ^"■^^'^ collected was another advantage 

ot the ARI, and a set of instructions for its use was provided. 

tPn.-,-'^'^'^'^^^r^^^°P^^^^^ ^^^"^ P^^^^'^ °" readability research. An ex- 

slZlT Tf\°^ available readability measures was undertaken (Williams 

? ^""^f ^' 197^^ - --P°-t was prepared suinnarizing a varietj 

lis (S±ltlT% T evaluating and improving readability'of n,ateril 

als (Siegel, Federman, & Burkett, 1974). This work served as the foun- 
imoiovP^°'' ^ subsequent series of efforts, still underway, to develop 
improved measures of textual comprehensibility based upon analysis o^ 
key cognitive and psycholinguistic factors that contribute to difficulty 
ketrlgVAr^'T^f.'' (Siegel Lan.bert, & Burkett, .1974 and Siegel & Bur- 
tirf^.^n ^ . cognitive/intellective factors and six psycholinguis- 

tic tactors were identified that could serve as the basis for a new 
approach to measuring comprehensibility. Measures of these factors were 

and experimentally validated, and current work Is attempting to 
fur .hex refine these metrics with a view toward ultimate development of 
a i.ec of computerized comprehensibility measures. 

The early inters^sts in materials and readability also gen-rated a 
somewhat difrerent line of research into ways of reducing readinrS- 
mands through use of specially designed media, simplification of materi- 
to LT/ techniques. A systematized audio/visual approach 

to self-paced job training called Automated Apprenticeship Training (AAT) 
which -Ployed an easily operated portable sound/f ilmstrip tScUng it ^' 
vice was developed (Pieper, Catrow, Swezey, & Smith, 1973). The AAT 
fomat was specifically designed to reduce dependence on reading skills 
and It was sho.m to be highly effective for low aptitude trai^eL with-' 
Z^^S:^^ higher ability men going through the same trainS 
progr^a Th^ AAT approach showed good potential as a means of reducing 

vlsuJ^tTr ''^^'f °" marginal airmen, and its ' self-paced audio/ 

^isuax teatures also produced substantial training time savings, with 
performance equal to or better than conventional Approaches. lather 
attempt to systematically reduce reading demands, a series of experiments 
was conducced to examine the effects of lo.vering the reading grade ^eve^ 
of textual material and providing tape recordings of the text'for in- 
creasing comprehensibility (Siegel, Lautman, & Burkett, 1974). Greatest 

of'tJe wr^tte . " f ""'^''^^ ^-'^^ ^^-^ difflcul y 
rLS thf r '^'^ """^ effective for self-study mate- 

rials than for resident course study guides. This study demonstrated the 
value of matching the difficulty of materials to the reading abUity of 
the men, but did hot show any particular advantage for additional audio 
supplementation of the material. ^ r^ - ^aaitionai audio 

1 U D 



110 



By 1973, there was an increasing Air Force. indication that the read- 
ing proolerns of some airaea were causing problems in both resident train- 
J-Ug ciia on-the-job training. Accordingly, studies of the reading abili- 
^-T"" f^'lS? °^ reading training problem were under- 

"t r* ^"""t ^^^^^^^ surveyed Air Force reading improvement programs 

and found that from 1 April 72 to 1 April 73, 5744 airmen participated - 

such programs, with the most frequently cited problem^ involving in- 
...ility to mad and pass CDCs. He also found that ten career fields 
accounted fox over 80% of the problem readers who had required the train- 
r::;, ^ mduded Aircraft Accessory Maintenance (42XXX) 6.2%, Air- 
craft Maxntenance (43XXX) 18.7%, Mechanical/Electrical Trades (54XXX) ' 
Structural/Pavements Trades (55XXX) 8.0%, Transportation (60XXX) 
IT" (^2X^0^) 5.9%, Fuel Services (63XXX) 5,U, Supply 
( 8.3^ Administration (70XXX) 12.7%, and Security Police (8i:iXX) 

^.-■.o. In a related study, Mockovak Xl974b) demonstrated a nethodology 
.or determxnins the reading skills and requirements of Air For^e career 
:Z Identifying areas where literacy gaps existed. Basically, 

t... i^ethodology involved conducting a readability analysis of a sample 
or Che reading ability of men in the career field using the Madden-fupes 
conversion from aptitude scores procedure previously mentioned in thi- 
review 3y computing differences between the readability of the materi- 

^;t'f ''^^'^^'"^ ^^^^^'^ f lit^^^^y g^PS could be determined 

ana reading requirement levels derived. 

At about this same time, the reading skills acquisition process was 
c.-.aiir.ed through research on a developmental model of auding and reading 
SKxxls (Sticht, Beck, Hauke, Kleiman, & James, 1974). The model suggested 
...a. rsaaing was based upon and utilized the sane c6nceptual base and lan- 
1 "'^^ in- auding (i.e., comprehending spoken language) 

tnat aoility to comprehend by reading could be improved by improving 
tna conceptual base and language skill by means of auding. Implications 
t...e also found for literacy diagnostic testing, in that it was-apparent 
that sone_reading problems may really be the result of poor language 
skills. . -t was. thought that the latter might be indexed by means of an 
auding test. Then, differences between a reading and language skill 
indexed by .the auding test, might be useful in prescribing reading or 
language training for individual airmen. A follow-on effort is now under- 
Wet}' to develop such tests. 1 

The personnel and materials areas were later integrated into an 
extensive effort designed to determine the reading requirements across 
Air rorce career fields and to identify problem areas where literacy gaps 
exisced Mockovak •(1974c), using the methodology he had developed earlier 
compiled reading and reading requirement data for 56 career ladders and ' 
reading ability target data for all 277 career ladders in the Air Force 



■'"See the paper by Groff in these proceedings. 
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Across the 56 ladders, averagr^^ reading requirement level (RRL) obtained 
was 12.3 reading grade level, ranging from a low REL of 10,6 for cooks 
to a high RRL of 14.0 for telephone switching equipment repairmen. 
Average reading ability of personnel across all 277 ladders was 10.8, 
but there x.;as considerable variation between ladders, with some ranging 
as low as 8.5 and others as high as 14.5. However, meaningful tarr;"^t 
levels for writers were obtained, and these are currently being v^seo in 
the preparation of Air Force training and job materials, replaoin;^ the 
older Hadden-Tupes data that had been in use since the mid-six ties. An 
average literacy gap of around two reading grade levels was ahjo found 
for the ten problem career fields previously noted, suggesting that such 
mismatches were, in fact, contributing to the Air Force's reading problems. 

VJork is currently being conducted on both readability of materials 
and on reading skills of personnel. Improving readability is the goal 
of present attempts to develop new textual comprehensibility measures 
(Siegel & Wolf, 1975). Reading skills of personnel are being addressed 
through development of literacy assessment tests for improved diagnosis 
of individual reading and auding problems and by efforts to develop and 
demonstrate prototype job-oriented reading training programs. 

. It: ir> anticipated that future research in the AFHRL program will 
ccntinae to follow a '^systems approach'*. That is, it is felt that if 
h^iadway is to be made, systematic research on major aspects of the lit- 
eracy problem from both a personnel and a materials viex>;point should be 
pursued. In dealing with the personnel, research is anticipated to 
address the problems of identifying better ways to measure their literacy 
^jk.Llls and of developing training techniques designed to upgrade their 
reading skills in a job-meaningful manner. Similarly, in the materials 
area, major thrusts are anticipated to involve both research on measure- 
ment of the comprehensibility and reading demands of job materials and 
on development of improved methods for preparing and modifying materials 
and media to appropriately useable literacy levels. 

CONCLUSION ' . 

The phase-out of Project 100,000 and recent increases in enlistment 
standards have eased the literacy problem considerably, but recent staff 
studies and research findings indicate that the Air Force still has lit- 
eracy and technical writing problems of sufficient magnitude to be of 
some coacem. Present indications are that the advent of an all-volunteer 
force has neither resolved the problem nor made it particularly worse 
(Vitola, Mullins, & Brokaw, 1974) as some had originally supposed it might. 
However, a major policy change or . mobilization such as .those that have 
occurred in the past could again rapidly intensify the problem. ;^ile the 
prior R&D accomplished by the Air Force on literacy, technical x>rriting, 
and related matters has provided a useful technology base and a variety 
of useful applications in these areas, no panaceas to the problems have 
been found. Gaps in our knowledge and techniques still' exist which should 
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be addressed to , provide the tools required to deal with problems in these 
areas, and futal A.ir Force literacy and technical '\>nri ting R&D will be 
oriented toward these technology gaps and toward those specific priori- 
ties and requirements for applied research in these areas that emerge in 
the years to come. 
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• COMMENTS ON THE PAPER BY BURKEH 

^, John Guthrie 

International Reading Association 

Guthrie first reviewed the highlights of Burkett's. paper, beginning 
with Burkett's statement that the basic problem which has been the focal 
point for much of the Air Force's literacy and readability research is a 
literacy gap between the rc^ading ability of persorJiel and the reading dif- 
ficulty of materials, Burkett then reviewed work lin readability of mate- 
rials and literacy training in the 1950s, pointing out that neither re- 
search on readability nor on literacy training had indicated major impact 
on people^s ability to perform their jobs. A second wave of studies con- 
cerned, personnel factors related to job performance and showed that high 
school graduates had higher -success rates than rion-high school, graduates, 
leading the Air Force to recruit high school graduates. However, Project 
IOO5OOO produced an influx of lower aptitude personnel, and once again a 
proliferation of research on how to best train and utilize such personnel 
occurred. This work included the development of job performance aids to 
reduce cognitive and literacy demands of jobs and job training programs. 
Other work, including the current work, focuses on the one hand on how to 
circumvent the person's literacy problem by reducing the reading demands 
of jobs or improving the readability of material and on the other hand on 

. the assessment and improvement of the literacy skills of personnel- through 
improved literacy training. In this regard, Guthrie considered the Air 
Force's and other Armed Services* literacy problems, as analogous to that 
faced by some developing countries. In such countries there is a literacy 
gap: a mismatch between the capabilities of the people and the demands 
of a modem agricultural or technological so.ciety. It is a goal of under- 
developed countries to develop their economic and social stature, and lit- 
eracy training is viewed as a means to that end, rather than an end in it- 
self. Thus, to achieve social stature there may be a need to increase 
agricultural productivity- In order to increase agricultural productivity, 
there is a need for farmers who can read instructions for new fertilizers, 
\<iho can calculate the properties of mixtures, and who know enough to know , 
when and when not to apply the fertilizers. For instance, in one agricul- 
tural country, cotton is a cash crop; therefore, a major literacy program 
focuses on teaching people to read vrithin the context of cotton growing. 
Another teaches reading within the context of banana growing, and so on. 

. Without overdrawing the parallel between the Armed Services and the devel- 
oping countries, Guthrie suggested that in both cases there is a job to be 
performed — cotton growing or troubleshooting a radar set — as the 
primary interest. Also, performance of the job requires job knowledge in' 
addition to relevant cognitive skills, such as reading, and motivation' to 
perform the job. Thus, job performance requires job knowledge, reading, 
and motivation. Hence, training programs may require the teaching of all , 
three of these factors in some integrated form. • ^ 
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ONGOING RESEARCH AND DEVELOPMENT 
IN LITERACY TRAINING 
IN THE ARMED. SERVICES 
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INTRODUCTION 



Tho papers of the preceding section provide a perspective on past 
R&D efforts on reading and on the ways in which literacy training has ? 
been conducted within the Services. The present section sunmarizes the 
current situation regarding literacy training and ongoing R&D projects to 
understand and manage problems of literacy within the Sexrvices. 

In his review of the Navy^s ongoing program of literacy research, 
Tom Duffy discusses two major efforts: those aimed at providing' a data 
base to characterize the literacy problem in the Navy, and those which : - • 
examine tho Navy^s reading training programs and the people in them. In ' 
the first of these two efforts, the research investigated reading ability 
and its relationship to personnel characteristics (education, race, and 
scores on the ability tests given in the Navy Basic Battery Test). and to 
tha readability of Navy manuals. Related studies looked at the relation- 
ship between reading ability and two measures used in the Navy to measure 
job effectiveness: attrition from recruit training and performance in "A" 
School. 

In presenting the second effort, Duffy describes three current Navy 
and Marine Corps literacy training programs and then discusses the mean- 
ing of student reading grade level gains in terms of entry selection, 
racial and educational characteristics of the students, characteristics 
of the programs, gains in civilian reading programs, and the criterion and 
ultimate goals of literacy training programs. Finally^ he briefly de- 
scribes three experimental programs and research efforts to characterize 
the successful student in the Navy program in San Diego. 

The research and development program in the Army parallels that of 
the Navy in that one phase has dealt with the nature of literacy problems 
and a second phase has dealt with a literacy training program. John Caylor 
summarizes the Army program, which consisted of (1) an earlier, primarily 
research phase that sought to define the literacy problem in terras of 
actual job functional literacy demands, and (2) a later, primarily devel- 
opmental phase that produced a job functional literacy training program 
aimed at meeting those demands. 

The literacy problem has two sides, as Burkett has stated in his 
historical 'review of Air Force R&D. The Army program has also looked at 
both sides of that problem. To determine the job functional reading re- 
quirements of selected Array jobs, the research studied the relationship 
of general reading ability to measures of job proficiency, estimates of 
the readability of Army manuals in the selected job areas, and performance 
on job reading task tests. Tliese three measures we're in general agreement 
v;ith each other. The reading ability of Army personnel in Mental Categories 
III and IV was assessed directly by a standardized reading test. A correla- 
tion study showed that reading ability could also be assessed indirectly 
from AFQT scores. The results of the research phase showed that the read- 
ing demands of Array jobs far exceeded the reading ability of many Army 
personnel. 

Ill 
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The information on the nature of the literacy problem was used to 
develop a functional jliteracy program (FLIT) which provides job reading 
training using the ccjjntent and reading materials of the student's job area. 
One segment of the program stresses the application of the student Vs ex- 
isting general reading skills to specific kinds of job reading tasks, using 
actual job reading materials. Another segment uses specially prepared job-' 
oriented reading passages and exercises to improve basic reading skills. 
This program was imiilemented at all of the Army' Training Centers. 

Steve Grof f opens his survey of ongoing literacy research and devel- 
opment in the Air Force by looking at the current Air Force literacy train- 
ing. During basic military training, reading training is provided in Read- 
ing Proficiency Units, which offer general literacy training. After assign 
ment to a permanent duty station, reading training may also be obtained 
through the base education office. These ^-eading programs vary greatly. 
The high numbers of people enrolled in literacy training and the results 
of studies which have found a literacy gap between the estimates of job 
reading demands and the reading ability of the personnel in. those jobs all 
point to a definite literacy problem in the Air Force, despxte^high enlist- 
ment standards. 

Current Air Force R&D involves two major efforts. The first of these 
is the development of a Literacy Assessment Battery which can be used to 
identify people who would most likely benefit from further literacy train- 
ing. The Batter^'- yields an indication of the discrepancy, if any, between 
the person's auding (oracy) and reading (literacy) skills. The person 
whose oracy skills exceed his literacy skills is the most likely candidate 
for literacy training. Such a person has a greater command of language 
skills than his reading score alone would suggest, and literacy training 
could bring his literacy skills up to the level of his oracy skills. The 
second ongoing effort is a prototype job-oriented reading program for the 
Air Force. This program is based on the approach usfed in the Army. FLIT.' 
program, but it focuses on the higher level comprehension skills. 

Groff points out that no one literacy program^meets all literacy needs 
for all people. A total career education program would include literacy 
training, not as a one-time shot, but available throughout an airman's car- 
eer, in different forms for different needs. Sucli a program could include 
the Reading Proficiency Unit training to improve basic reading skills , the 
job-oriente(d reading program to improve conceptual behavior in a job con- 
text, and general reading/GED programs to provide access to higher levels 
of skill. 
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LITERACY RESEARCH IN THE NAVY 

Thomas M. Duffy 
Navy Personnel Research and Development Center 

■ - . \ 

. Literacy research in the Navy today has had two major objectives.' 

First, there have been efforts to characterize the literacy "problem" in 
the Navy and thereby provide a data base for evaluating altBmative ap- 
proaches to insuring that a literacy deficit at any level does not hamper 
the effectiveness of the service. The second major line of research has 
focused on the Navy's current approach to deficient literacy sk.llls — 
reading training programs for personnel with marginal reading ability. 
Each of these avenues of research are discussed below- 

CHARACTERIZING THE LITERACY PROBLEM IN THE NAVY 

In this\research, a group of us at NPRDC have initiated efforts to 
characterize the reading skills of personnel and examine basic demograph- 
ic data on reading and its relationship to other abilities and background 
characteristics. This research has also involved an examination of the 
relationship of reading skill to job performance. 

Interrelationships Among Reading 
and Other Personnel Char^acteristics 

In a joint effort with the Naval Training Center and Recruit Train- 
ing Command in San Diego, we have now collected reading and other ability 
and background data, as well as performance data, on some 25,000 recruits 
and 1-500 men receiving occupational ("A" School) training. 

For both the recruit and A School samples, we obtained all of the 
test and background data collected as a normal part of processing into' * 
the service. These data include the following test scores and background 
information: 

1. General Classification Test (GCT) — A test of general 
ability involving verbal analogy and sentence completion 
items . 

V 

2. Arithmetic Reasoning Test (ARI) - A test of mathematical 
ability involving, word problems. 

3. Mechanical Ability (MECH) - A low-verbal test of knowl- 
edge of mechanical principles. 

4. Clerical Test (CLER) - A speeded digit search test re- 
quiring no verbal skill. 

5. Electronics Test (ETST) - A verbal test of electronics 
aptitude* 
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6. Shop Practices Tes^t (SHOP) — A test of knowledge re- 
quiring the matching of a picture of a tool to verbal 
descriptions of uses. 

7. Armed Forces Qualification Test (AFQT) — A score de- 
rived from the GCT, AM, MECH, & CLER by- their regression 
on the former AFQT test of general ability which was ad- 
ministered throughout the Armed Forces. 

S. Years of Education — Self reported. 

9. Race — This data was obtained only for the recruit 
sample. 

In addition to the routine personnel data, all of the A School sample 
and 1,294 recruits were also tested on the Navy Pattern Matching (PM) test 
of nonverbal ability. The PM was derived from the Raven's Standard Pro- 
gressive Matrices Test. (Raven, 1958) which is considered by some to be a 
culture-free test of nonverbal intellectual, ability (Jensen, 1972; Carver, 
1973d; Spearman, 1946). " - 

Recruits' reading grade level scores were obtained using the vocabu- 
lary and comprehension subtests of the Gates-MacGinitie Reading Test, 
Survey D (Gates & MacGinitie, 1965). ' The A Gchool personnel were tested 
using the vocabulary and comprehension subtests of the Nelson-Denny Read- 
ing Test (Nelson & Denny, 1960). 

Reading Levels 

Figure 1 presents the distribution of reading levels in the recruit 
sample. The skewness of the distribution reflects the limitation of the 
Gates-MacGinitie test (which has a maximum score of 12.0 RGL) for this 
sample. 

The median RGL is ,10. 7, which is generally reflective of the educa- 
tion level in our sample, in which 70.9% left school after either the 11th 
or 12th grade. However, the distribution of scores indicates that 18% of 
our sample has less than an 8th-grade reading "skill. In order to get some 
idea of the implications of these reading scores, we did a readability 
analysis of those manuals and 'tests which most recruits are expected to 
read. The materials were analyzed using the FORCAST readability formula 
(Caylor, Sticht, Fox, & Ford, 1973). The materials assessed and their 
reading difficulty were as follows: the Bluejackets* Manual (11.5 RGL)," 
which is the basic manual in boot camp; the Airman (10-5), Fireman (10.2), 
and Seaman (10.2) rate training manuals used- during and right after com- 
pleting boot camp; and the general classification test (10.8 RGL) and 
arithmetic reasoning inventory (9.0 RGL) which are administered in the 
process of classifying recruits. 
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A comparison of the rea^iing and readability levels indicates that 
40% to 50% of the recruits have a reading ability less than that demanded 
by their job reading material. I^ile the effects of this apparent "lit- 
eracy gap" on comprehension and performance is uncertain, it stems likely 
that the 1.8% of recruits (a projected 1800 men for FY 1974) who read be- 
low the 4th-grade level will have great difficulty, if not total impossi- 
bility, in comprehending these job reading materials. Further, in our 
recruit sample, 18%, or a projected 15,800 men, entered the Navy during 
FY 1974 with reading skills below the 8th-grade level. These men may be 
expected to comprehend most Navy material if they can reread the material 
and are assisted by other personnel. However, the efficiency of their 
job performance will most likely be impaired and of a marginally accept- 
able level. If, in an emergency when time is limited, these men are re- 
quired to act in any way that requires the use. of printed material, they 
may well prove a hazard to effective Navy operations. 

The Nelson-Denny Reading Test was used in assessing the A School 
sample, since it provides norms for higher levels of reading ability. 
However, because of the change in reading tests, the recruit and A School 
data cannot be directly compared. Within the A School sample, the mean 
and median RGLs were 11.1 and 11.2 respectively, Evtm though the A School 
personnel are selected on the basis of- ability, we find that 9.6% of the 
sample had reading skills below the 8th-grade level. Table 1 presents a 
more detailed exaiuination of the reading levels in each of six A 'chools, 
as well as an indication of the reading requirements (readability of 
materials) in those schools. 

Tlie readability score indicated in Table 1 is for the rate, training 
manual which is studied for advancement to the 3rd and 2nd class rates. 
The manual is used in the A School classroom and the rate exam is typic- 
ally taken within three months of completing the school. The data in 
Table 1 indicate that for all but th^ Ship's Serviceman, the mean reading 
ability of the men approximates and in some cases exceeds the mean read- 
ing difficulty of the manual. However, in three of the six schools, a 
significant proportion of the men are low ability readers and ,may be ex- 
pected to face reading difficulties. Even in" those ratings where the 
proportion of low ability readers is small, the actual number of men, 
several hundreds^ per rating based on FY 1974 manning figures, is con- 
siderable. 

All of our assessment data indicate that the options available in 
approaching literacy deficits in the Navy will affect significant numbers 
of personnel. For example, if a 5.5 RGL, the current reading level deemed 
necessary in the Navy, were required for admission 'Into the service, 9% 
or 7,800 of the FY 1974 recruit population would have been rejected. 
Eighteen percent of the recruits or 15,800 annually would be eligible 
for a literacy program expanded to an 8.0 RGL terminal criterion, a cri- 
terion which only approaches the initial reading demands in the service. 
The same number of men would be affected by the implementation of limited 
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TABLE 1. READING GRADE" LEVELS CRGL) OF MEN & READING GRADE 
LEVEL REQUIREMENTS FOR MANUALS IN EACH OF SIX 
NAVY OCCUPATIONAL TRAINING "A" SCHOOLS. 



SCHOOL 



MEN . MANUALS 

_ ^ % BELOW _ 

X RGL 8.0 RGL X RGL' 



*« 



Hull Maintenance Technician 

Ship's Serviceman i 
Mess Specialist ■ 
Quartermaster •■' 
Interior Communications 
Electrician's Mate 



10.2 


19.1% 


10.7 


10.^ 


18.1% 


. 12.9 


10.5 


12.0% 


10.2 


11.7 


2.9% 


10.9 


11.9 


5.1% 


12.5 


12.0 


• 2.3% 


12.7 



*RGL for men is based on Nelson-Denny reading test perform- 
ance. Mean RGL is presented; means and medians were 
comparable in each school. 

■-^^Readabiljty scoj^es are taken from Biersner (1975) and are 
based on"^the application of the Flesch Reading Ease formula 
nonnsd on Navy fnen and manuals (KincaidTet al., 1975). 
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duty assignaancs for below-8Ch-grade readers. The costs involved and 
the number of men affected by implementation of any of these options 
demands that the options be carefully considered. 

Our findings ^^th regard to literacy skill levels in the Navy indi- 
cate that one or more of the above options must be implemented to assure 
effective performance levels in the service. These options, however, 
cust necessarily focus on the very low literate man in order to be man- 
ageable and within reasonable cost figures: The distribution of reading 
skills in the A Schools suggests an additional option which would address 
personnel with less severe literacy deficits. This option involves a 
literacy training program for personnel deficient in reading skills, but 
operwise qualified for A School. The program would have as a CA^QjUon, 
-diz cbU^y dzmancU zach school and moM pfiovldz ^uncXlonal /LQ.acUng 
■iJicu.i-u.Kg preparatory to, or in conjunction with, school training. The 
major benefit of this program is that it would provide the opportunity 
of upward mobility and career fle:<ibility for men who, while eligible for 
A School training, are likely to perform at a marginal level. Addition- 
ally, the program would involve fewer personnel and hence, would be more 
manageable than a broad-based literacy program. ' HumRRO, under contract 
to the Center, is currently evaluating the cost-effectiveness and man- 
agement considerati^ons involved in this and other literacy training 
options. 

Education. It is frequently asserted that the current move toward 
accenting only high school graduates will greatly reduce or eliminate the 
marginally literate personnel, and hence, there will be no need for Navy 
commitment to literacy training. Our data indicate, to the contrary, 
that a high school diploma bears little relationship to reading skills. 
The median reading levels for our high school and non-high school gradu- 
ates dxffer by less than one RGL (the medians are 10.9 and 10.2 respec- 
tively). Figure 2 presents the distribution of reading grade levels. 
While proportionately fewer high. school graduates fall below an 8th-grade 
RGL, this proportion (16.9%) nonetheless indicates that the selection of 
only high school graduates would still produce a significant number of 
marginally skilled individuals. 

Further analysis on a sample of 19,000 recruits indicates that years 
of education, ranging from 3th to 6th grade, correlates only 0.13 with 
reading ability. Of all of the test and background information for which 
v/e haVe data, years of education shows the least relationship to reading. 
This finding may be contrasted with previous research which has found 
years of education to be the most valid predicto. of attrition from the 
service (Flag & Hardacore, 196'f>) and delinquency (Cunderson h Ballard, 
undated). One might conclude from these data that a high school diploma 
indicates a person's willingness to conform to the rules of society rather 
than his ability. However, a more likely explanation is that over recent 
years the predictive power of years of education has been reduced due to 
the increasing porpdrtion of high school graduates entering the Navy. , 
Evidence for this hypothesis comes from the additional finding in our 
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analyses thaVyears of education is no longer a strong predictor of 
attrition. The reduced variance. in years of education is indicated by 
the fact that 77% of our sample completed 11 to 13 years of education". 
Even though the lack of a significant correlation between education and 
^reading likely rests in the restricted variance in education, it. is an 
interesting reflection on our educational system that die difference in 
reading level between high school graduates and nongraduates is minor^ 
Clearly, tha data in Figure 2 indicate that a move toward accepting only 
high school graduates will have little effect on reading skills in the 
Nnvy. 

Race. Figure 3 presents the distribution of reading scores for each 
of the three major race categories at the San Diego Recruit Training ' 
Center (RTC) . Approximately 1-1/2 RGLs separate the median scores of the 
gr^'nvr-, with Caucasians having the highest median (11.0) and Malaysians 
t. ... lowest (7.6). The same ranking of r.aces is obtained when one com- 
petes the proportion of men reading below the 8th-grade level. However, 
in terms of absolute numbers, the men with lower reading skills at the 
San Diego RTC are primarily Caucasians and Malaysians. 

The race data suggest a major cause of many of the reading problems 
foremen at the San Diego RTC. The Malaysians, while representing only 
8.6% of the recruit population, constitute 28,0% of the population of 
below-8th-grade readers. The Malaysians also constitute a grcup in which 
English is a second language for most men. Although English is taught 
beginning in the first grade. in these countries, its use is restricted 
almost entirely to the school and is limited to '^formal English", Thus, 
a deficiency in functional reading skills may be expected. 1 

The relationship betx^/een reading stcill and racial category indicates 
that any policy of selection. or classification on the basis of reading 
skills x^ould have definite effects on the racial distribution in the Navy. 
The data also indicate that any reading training program will have to have 
the capability of dealing with skill deficiencies characterizing personnel 
with English as a second language, as well as personnel with English as a 
native language, but who have' failed to develop adequate reading skills. 

Ability Tests . One of our primary interests in the ability test data 
was to determine the degree to which reading abili;:y could be predicted 
from performance on standard Navy tests. Our testing program was limited 
to San Diego and was due to terminate in March of 1974. A continued track- 
ing of reading levels in the Navy, continued determination of low ability 
readers, and determination of reading levels at the other RTCs would be* 
possible if a strong relationship were obtained between reading and the 
Basic Battery Test. 



The race data are not necessarily representative of the Orlando and Great 
Lakes RTCs, since the large proportion of Malaysians at San Diego is due 
to its geographical location. However, considerations of .English as a 
second language may apply to the segment of Spanish-speaking people found 
at those training comiriands. 
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Data from the first 7093 recruits taking the reading test were used 
m the initial analysis. All Basic Test Battery scores were entered as 
prediction variables into a stepwise multiple regression analysis with 
the Gates-MacGmitie reading score as the dependent variable. The GCT 
a highly verbal general ability test, was found to be the best predictor 
of reading, r = 0.73 and standard error =. 1.36 RGL. The clerical test 
entered next into the regression equation, but added only 0.006 points 
to the multiple correlation. Thus, the prediction of reading is most 
efficiently based on GCT alone. These findings are similar to '.the read- 
ing - general aptitude correlations of 0.68 and 0.82 obtained'in the 
iZl^^""^ t^"" respectively (Caylor et al. , 1973; Madden & Types, 

iJbb). The degree of relationship is to be expected since the general 

^"^^^ ^^""^ ^° ^^S^ly ^^^bal tests, as was described for 

The empirically determined RGL, as well as the predicted RGL for • 
each ^ level of GCT, is presented in Figure 4. The empirical relationship 
IS linear m the midranges, but both a basement and ceiling effect are 
evident. The ceiling effect is due to the limitation of the readin- test, 
wnile the basement effect is likely due to a lack of sensitivity of °the 
ULx at the low end, since the GCT v;as designed to predict school sii'ccess 
ratner than to discriminate between low ability personnel. 

The results of these analyses (and o^i.r^, including a cross-valida- 
tion study) indicate that GCT can be used. V crack changes in reading > ' 
Skills m rt:r Navy's assessions. In addition, GCT can be used tc obtain 
on approximatiuii of proportion of recruits falling into gross categories ' 
nf reading skill. However,. OCT cannot be used to discriminate between 
low ability readers. At best, the: results indicate' that 95% of those 
recruits reading below the 6th-grade level will have a GCT less than 45. 
In the future, we will be comparing this regression equation with one ' 
derived from the Nelson-Denny test in our A School sample. In addition 
to these regression analyses, we are currently using the data base pro- 
viaed by the remaining ability tests described earlier to evaluate vari- 
ous hypotheses, concerning relationships of reading ability and nonverbal 
intelligence. 

'Reading Ability and Navy Effectiveness 

A basic assumption underlying the interest in' literacy in the services 
and the very occurrence of this conference is that reading ability is a 
significant factor in determining the effectiveness of a man in the service 

Logical arguments relating reading ability to job performance have 
been presented by Duffy, Carter, Fletcher, and Aiken (1975) and Sticht 
(197^a). However, research in support of the proposition is meager. Those 
studies which have examined the reading/ job performance relationship have 
typically had direct reading skill measures for a "low reading" group 
but have inferred reading skill from general classification test perform- 
ance for a low ability" comparison group (e.g.. Fisher, 1971; Hagen- & 
Thorndike, 1953; Hoiburg, Hysham, & Berry, 1974; Standlee, 1954). Our 
data indicate that reading and general classification test performance 
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10 - 
9 - 
8 - 
7 - 
6 - 
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^ - 
3 - 
2 - 




CORRELATION = .73 
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95% 

CONFIDENCE LIMITS 



20 



30 



^0 50 60 
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Figure 4. Relationship Between GCT & Reading Grade Level 
for 7093 Recruits. 
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are not highly correlated ia.the low ability personnel. The relationship 
between reading and performance in these studies, therefore, cannot be 
accurately determiuGd, Even if reading ability had been directly assessed, 
these studies are limited in focus to personnel bordering on functional 
iliitei'ac.y. Thus, if no relationship was found between -^reading and per^- 
fonnance in this low-le\el population, this may simply Indicate that all 
of the Tuen were so deficient in reading skills that variation in skill 
level ware irrelevant to perf orn?.ance . That is, even the highest reading 
score repv&Sc-nted inay have failed to reach the threshold required for 
producing increments in job performance. 

Our recruit and A School samples provide the opportunity to examine 
the relationship between reading and performance where the full range of 
reading skills is directly assessed and represented. Our interest in 
assessing this relation was both, to examine the contribution of reading 
to performance and also to determine the merits of using reading .ability 
in the selection and classification of recruits, A problem with this type 
■of research is that performance measures tend to be restricted in range 
to scores around that point defined as "adequate" and therefore, the mag- 
nitude of the correlation to be expected is limited. Additionally, the 
factors determining a performance score differ depending on the purpose 
of the score and the nature of the assessment. Different types of per- 
formance scores do p,ot always reflect similar performance levels. For 
example, Sticht, CayAor, Kern, and Fox (1971) found that a job sample 
test measure of job performance correlatec' only 0.13 to 0,24 v7ith a sup- 
er\dsor's rating of performance, Ronan and Prien (1971) provide a de- 
tailed discussion of measures of performance. 

With these difficulties in performance measurement in mind, we have 
examined the use of r reading in predicting performance on two measures 
currently employed in the Navy to measure job effectiveness: A School 
performance and attrition from recruit training, Prediction of these 
performance measures has been the subject of considerable research, since 
projected attrition and A School performance are fundamental to the selec- 
tion and classification of personnel. The-contribution of. reading to the 
prediction of these effectiveness variables would indicate both the im- 
portance of reading in the Na^/y and the usefulness of reading skill level 
as a selection or classification variable. 

Reading in Relation to Attrition from Recruit Training 

The sample of recruits for this analysis consisted of 22,840 men ' 
entering RTC, San. Diego, between 1 June 1974 and 1 February 1975. The 
dependent variable was attrition from recruit training. The attrition 
lists at the RTC, San Diego, were checked monthly and all attritees for. 
whom we had reading data were noted, along with the reasons for attrition.. 
Three predictor variables were used: 

1. Odds for Effectiveness (OFE) — An estimate of the proba- 
bility of completing the. first tour in the service and being 
recommended for re-enlistment. This is an actuarial table 
used by recruiters in which a composite score is derived 
from years of schooling, number of expuls^ions and suspensions 
from school, and Armed Forces Qualification Test Score (Plag, 1968). 
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2, Armed Forces QualijCication Test (AFQT) — This is a score 
' • derived from performance on the GCT, ARI, and MECH tests 

taken at the recruit station. Placement into a mental cat-* 
egory is based on this score. 

3- Reading Ability — Performance on the Gates-MacGinitie 
Reading Test described previously. 

Results indicated that in our total sample of recruits, there was a 
7.7% attrition rate as compared to an overall attrition rate at Sah Diego 
of approximately 10% during our test period. This disparity reflects the 
attrition of personnel prior to administration of the reading test, as 
well as personnel who were hospitalized at the time of testing and even- 
tually attrited. The distribution of reading ability for the attritees 
and nonattritees in our total sample is shown in Figure 5. The primary 
reasons for attrition in our sample were inaptitude (academic and mili- 
t^^y) , physical disability, and psychological disability. Approximately 
30% of the attritees were in each of these three categories. Even x^th 
this wide variability in the reason for attrition, the data in Figure 5 
indicate very different distributions of reading skills for attritees and 
nonattritees (median reading lavels are 8.2 and 10.9 respectively). Tne 
probability of attrition in each of the reading categories is sliovvn in 
Table 2. A clear and systematic relationship between reading ability and 
the probability of attrition is evident. The less-than-4th-grade readers 
have a 0.36 probability of making it through boot caiiip, while the proba- 
bility is 0-96 that the above-lOth graders will make it. 

While reading ability is clearly related to attrition, the question 
still renains as to the independent contribution it makes relative to 
other available indices. Tc answer this question, we entered the readings 
OFE, AFQT, and years of education scores into a stepwise multiple regres- 
sion analysis with attrition, a dichotomous variable, as the dependent 
measure. The multiple regression analysis of the initial sample yielded 
reading as the strongest predictor of attrition (r = 0.25). The addition 
of the remaining three predictor variables added only 0.,02 to the multiple 
correlation. The reading-attrition relationship increased to 0.33 in a 
cross-validation sample, and here the remaining three variables added only 
0.003 to r. The independent correlations of OFE, AFQT, and years of edu- 
cation VTith attrition in the initial sample were 0.19, 0.17, and 0.14 
respectively. The comparable correlations in the cross-validation sample 
were 0.16, 0.18, and 0.12. 

In summary, these attrition data indicate that attrition rate system- 
atically decreases as reading ability increases, and this holds through 
the full range of reading levels. Reading is the best single predictor 
of attrition from boot camp. The degree of relationship between reading 
and attrition is significantly larger than the correlation of attrition 
with those measures currently used by recruiters to predict the successful 
sailor. VThile this latter finding indicates that JtZadLing COYitXibuitQJ^ iZq- 
yiiilcxL^AXlLj to tiiz p/L^diatLon cuU/utlon Znd^pzndQyitZy tiid oikvi vcuvi- 
abZu , we performed an additional analysis to confirm this conclusion. " 
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•A stepwise regression analysis was performed on the total data sample* 
This time, however, OFE, AFQT, and years of education were entered into 
the analysis before reading. These three variables together yielded a 
multiple correlation with attrition of 0.20. The addition of reading 
as a predictor raised this correlation to 0.30,. That reading ability 
as measured by the Gates-MacGinitie test contributes to our prediction 
of attrition over and above other tests involving reading and non-reading 
components (APQT) , as well as amount of education, strongly suggests that 
it is reading skill with its various components (e.g., langxiage, semantic 
knowledge) per se and not other abilities underlying reading test per- 
formance (such as perseverance to complete high. school, conformity to 
expectations) which is related to attrition. Since the primary emphasis 
in boot camp is acculturation and does not involve excessive use of man- 
uals, the relevance of reading skill to performance after boot camp should 
be even greater. With regard to selection policies in the Navy, these 
data indicate that use of a reading test would yield more accurate pre- 
dictions of attrition than the measures currently in use. 

Reading in Relation to A School Performance 

The A School sample and the predictor variables are as described in 
the previous section. The dependent measure for this portion of the 
project was average performance on a weekly paper-and-pencil test. Sub- 
sequent analyses will examine an "in school*' job performance measure, as 
well as the final score in the school, which is a score based on all pre- 
„vious-_te sting.. 

Only a preliminary analysis of the relationship of each of our pre- 
dictor variables to weekly test performance has been completed. In this 
analysis, we were again interested not only in the degree of relationship 
between reading and performance, but also how that relationship compared 
to the largest correlation between one of the current predictor tests and 
performance. The data relevant to these considerations is presented in 
Table 3, where it can be seen that the reading/performance correlations 
range from ,0.20 to 0.50 -across the six schools, while the maximum pre- 
dictor test correlation with performance ranges from 0.21 to 0.61. 

Additional analyses are planned using multiple regression techniques 
to further evaluate the usefulness of a measure of reading ability for 
predicting A School weekly paper-and'T)encil test scores. If lower ability 
readers are poor risks in the A Schools, as suggested by the data of Table 
^3, an A School preparatory reading training program might be instituted 
to provide the deficient readers with the necessary skills for successful 
performance in their technical training. 
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TABLE 3. CORRELATION OF READING, PAHERN MATCHING, AND THE 

MOST PREDICTIVE TEST IN THE BASIC TEST BAHERY (BTB) 
WITH WEEKLY TEST AVER.AGE- IN SIX NAVY "A" SCHOOLS. 



WEEKLY TEST AVERAGE CORRELATION WITH: 







. PATTERN 


BTB 


SCHOOL 


READING 


MATCHING 


TEST 


HULL miNTENANCE TECHNICIAN* 


.20 


.05 


.21 (ARI) 


SHIP'S SERVICEMAN 


.- .26 


• .16 


.21 (ARI) 


MESS SPECIALIST 


A3 


.22 


.32 (ARI) 


QUARTERMASTER ■ • . 


.50 


.^18 


.61 (ETST) 


INTERIOR COMMUNICATIONS 


.39 


.39 


.50 (ARI) 


ELECTRICIAN'S MATE 


.23: 


.16 


.^4 (ETST) 



A WEEKLY TEST. IS NOT ADMINISTERED. IN THIS SCHOOL AND THEREFORE THE 
DAILY TEST AVERAGE SCORE WAS USED 
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LITERACY TRAINING 



I would like now to narrow the focus of consideration to personnel 
demonstrating major deficits in reading skills — the below-6th-grade 
reader. The Marine Corps and the Navy provide reading training during 
recruit training for these low literate men. .-The- obj ectives of the 
training are (1) to provide a level of literacy skill to all personnel 
so as to ensure fleet effectiveness and fleet safety, and (2.) to provide 
the literacy skills necessary for equal opportunity in attaining upward 
mobility and a successful career (Steward, 1974; Academic Remedial Train- 
ing, 1975). Clearly, data on the reading requirements in the Navy pre—, 
sented earlier and described in more detail by Carver (1973b) and Curran 
(this conference)^ suggest that the training criterion, roughly a 5th-grad( 
reading level, is' not adequate to achieve either of these objectives. 
Unfortunately, the data required to evaluate whether or not the objec- 
tives are being met has not been gathered. However, there is data rele- 
vant to the more immediate objective of the literacy training; that of 
improving reading skills. In this section, I will describe the effect- 
iveness of three literacy programs in achieving this more immediate 
objective. In addition, I will describe three experimental training pro- 
^ grams which have just been initiated. Finally, I will describe my o\>m 
research efforts to characterize the reading deficiencies and to predict 
the probability of success in the current training programs. 

Current Progrdms 

The three programs to be described are Navy programs in San Diego, 
California, and Orlando, Florida, and a Marine Corps program in San 
Diego, California. Program descriptions and evaluation data were ob- 
tained from the 'follo\^^ing sources: Marine Corps (Stewart, 1974, 1975); 
Navy in San Diego (Academic Remedial Training, 1975); Navy in Orlando 
(Research for Better Schools, 1974). 

The Orlando program was an experimental effort to test the Individu- 
alized Learning for Adults package developed by Research for Better 
Schools (RES Program). This is a self-paced p-rograr;jt for illiterate 
adults and provides training material to the 9th-grade level. Training 
was on general 'literacy and focused on the specific skills of recognition 
of sound-symbol relations (phonics), word attack (vocabulary), compre- 
hension (literal and interpretive) , and study methods. There are 129 
performance objectives in the program with 32 of the objectives devoted 
to decoding skills, 21 to -word attack, and 41 to comprehension skills. 
Five entrance tests were used to determine the appropriate starting level 
for an individual and performance during training was regularly assessed 
using 70 different pre- and por? t-tes ts. The training materials were 129 
study booklets and ,54 audio cassettes specially prepared for this pro- 
gram. The maximum student-instructor ratio was 7 to 1^ (4 instructors and 
a maximum of 28 students in a classroom)* The instructors were Navy men 
selected on the basis of having a college degree (area of study unspeci- 
fied) . Instructional time ranged from 10 to 160 hours with a mean of 
60 hours. 
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A total of A3 male recruits participated in. the RBS program between 
September 1973 and February i974. The criterion for selection was a score 
of 7.0 RGL on the combined vocabulary and comprehension subtests of the 
Gates-MacGiniti^ Reading Survey D. The methods of selecting recruits to 
test were not specified. After three weeks in the program, the student 
was retested in an alternate form of the Gates-MacGinitie test. If this 
score exceeded a 7.0 RGL, the man was returned to recruit training; if 
under 7,0, he continued through the entire RBS program. 2 

' The Marine Corps (MC) program in San Diego was devised especially 
for the Marines, in conjunction with the local adult school. This is the 
only program involving civilian instructors, all of which are female and 
hold teaching credentials and adult school certificates. The program 
involves some self-pacing, but is classroom-based and fixed at four weeks 
(120 hours) of instruction. The training is in general literacy and in- 
structional time, according to the program syllabus, is distributed as 
follows: decoding (phonics), 40%; reading speed and comprehension, 42%; 
vocabulary development (sight vocabulary and word attack), 15%, Thus, 
there is considerable emphasis on decoding skills in this program, A 
wide variety of commercially available training materials and diagnostic - 
testis are used in the program, including the Controlled Reading Skill 
Development Series (Educational Developnient Lab) , McCall-Crabbs series 
(Columbia University), Reading Attainment: System (Grolier Education Corp,), 
Reading for Understanding (Science Research Associates) , and Phonic Word 
Blend Flip Char ts- (Kenwor thy Educational Service), 

All recryits at the Marine Corps Recruit Depot in San Diego who read 
below the 4,5 grade level are accepted into the MC program. Initial 
screening is accomplished through the administration of the Gates-Mac- 
Ginitie Reading Survey D to all recruits scoring below approximately the 
70th percentile on the general classification test. Recruits scoring be- 
low the 4,5 RGL are retested on an alternate form. of the Gates-MacGinitie 
and only if they onCe again score less than a 4,5 RGL are they admitted 
to- the program. The score on the second test serves as the person's en- 
trance score. Evaluation data are available on 492 recruits entering the 
program between February 1974 and July 1975. ' 

The Navy program in San Diego (NSD program) is the only program of 
the three under discussion which is run entirely by the service. Instruct- 
ors are Navy personnel having a college degree and the training materials 
are selected or developed by these instructors. Approximately 25% to 30% 
is devoted to phonics training. The phonics materials are based on the 
Motts Phonic System (unreferenced) and are- similar to those used in the 
MC program except that phonics is concentrated in the first week of train- 
ing. Successive weeks of the program deal with vocabulary (60 words per 
Xveek) , comprehension, and reading speed. The program is lock step and is 
a minimum of 3 weeks. Failure in any week results in repeating that week. 
Thus, the program averages 4*W2eks (92 hours) and ranges" from 3 to 6 weeks, 
2 

"The fact that a student was not retested until 3 weeks into the pro- 
gram appears inconsistent with the report th^t some men completed 
.the program in 10 hours. No explanation i? available. 
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Beginning in July of 1974 (ET 1975) specific testing for knowledge of 
sound-symbol relations was instituted. If skills in this area were 
adequate, the recruit bypassed the first week. Thus, in FY 1975 the 
ayarage program duration vas reduced to somewhat less than three weeks 
(69 hours) mth a range of two to five weeks. 

The training materials in the NSD program are a mixture of instruc- 
tor-generated worksheets and commercially available supplemental mate- 
rials. Hardware is lim^. to reading packing devices. Prior to FY 1975 
the supplemental materials consisted of the McCall-Crabbs readers and a 
few magazines (e.g.. Scholastic Hagazine). In FY 1975 a wide variety of 
connnercially available training packages, books, and magazines were intro- 
duced into the program. It is estimated that in both FY 1974 and 1975 
approximately 70% of the vocabulary training material and more than 90% 
of the comprehension training material focused on general literacy. 3 . 

The NSD program is directed at recruits with an RGL between 3.0 and 
5.5. ?rior to FY 1975 all recruits failing the first, academic test in 
recruit training were administered the Gates-MacGinitie Reading Survey D. 
If a recruit ^cored below the 3.0 level he was recommended for discharge. 
Recruits between a 3.0 and 5.5 RGL were admitted to the NSD program. In 
i'T 1975 the Gates-MacGinitie test was administered to all recruits. 
Those scoring less than 5.5 RGL were then retested on an alternate form. 
Only those scoring between 3.0 and 5.5 RGL on the retest were admitted 
into the program. 

The distinction between FY 1975 and FY 1974 procedures is important 
in the consideration of - the effectiveness of the programs. The RBS pro- 
gram and the NSD program prior to FY 1975 admitted students on the basis 
of a single test administration. Since only low scoring individuals were 
admitted into the program and later retested, the change scores should 
be subject to considerable upward regression.. The MC program and the 
FY 1975 NSD program, by selecting students after two test failures, should 
reduce this regression effect. Thus, the regression effect should in- 
crease the amount of artifact in the gain in the FY 1974 NSD and RBS pro- 
grams to a greater degree than in the other two programs. 

In addition to the program characteristics described thus far, dif- 
ferences in student characteristics and program policy may be expected 
to affect gains in reading within the programs. Tables 4 and 5 summarize 
the racial and educational characteristics of recruits entering each of 
the programs. The racial distribution is about equivalent in the MC and 
NSD programs except for an increase in the proportion of Malaysians (and 
thus, second language training requirements) in the FY 1975 NSD program. 
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The NSD program is currently undergoing another revision. The new mate- 
rials are estimated to be 70% to 80% Navy-related and the instruction 
will be individually paced. 
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TABLE i| 

PERCENT OF RECRUITS IN EACH RACIAL CATEGORY FOR 
FOUR READING TRAINING PROGRAMS 



RACE 



PROGRAM 


CAUCASIAN 


BLACK 


MALAYSIANS 


OTHERS 


BBS ' 










MC 


45 


29 


11' 


15 ■ 


FY74 NSD 




26 


12 


18 


FY75 NSD 


HO 


20 


26 





* RACE. DATA UNAVAILABLE FOR THIS SCHOOL 



TABLE 5 

PERCENT OF RECRUITS AT EACH LEVEL OF EDUCATION FOR 
FOUR READING" TRAINING PROGRAMS 



EDUCATION LEVEL 



PROGRAM 


8 


9 


10 


11 


12+ 


RBS 


7 


16 


26 


■ 30 


21 


HC 


n 


21 


25 


25 


. 18 


- ¥Y7H NSD 


2 


12 


.. 15 


25 




FYZS NSD 


3, 


6 


15 


23 


53 
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Educational differences, how&ver, are considerable. Approximately 50% 
of the NSD students have a high school diploma, while only 20% of the 
students in^the other programs are graduates. If a higher level of edu- 
cational achievement is indicative of the acquisition of better developed 
s.udy skills, then the NSD programs may show greater reading gains due 
sxmply to the relatively greater level of eductional. achievement of their 
otudents. Parenthetically, it is interesting to note that the data in 
iabie 5 reflect the conclusion drawn earlier that a high school diploma 
does not ensure that a person is functionally literate. 

Table 6, while summarizing instructional gains in the programs, also 
contains information relevant to our consideration of the relative gains 
one might predict for the programs. First, the entry level reading scores 
nof .IZ those in the other programs. The values were 

not subjectea to statistical comparison because of lack of adequate raw 

. °^ ""^^y varying Ns. However, the entry scores fall 

"""i ^^T^'^^L''^'"^^ °^ ability at which phonic skills are acquired. Thus, 
a 1 RGL difference in entry levels should reflect a meaningful difference 
m phonxc skills, which are considered by ' instructors as the most diffi- 
Zll A ^^°^-^°'^s^i'^g skills to acquire. In a post hoc vein, our own 
data and data from the MC and NSD programs indicate that entry reading 
level^is a good predictor of amount of gain (a positive relationship), 
iherexore, on the basis of entry level scores, the MC, program should yield 
smaller gams. Table 6 also indicates that the NSD progrLs attrite a 
considerably greater proportion of students.. Since only graduates are 
entered into the post-test calculation, this greater attrition is an arti- 
tact which would increase the gain in reading skills found in the NSD 
programs. 

Table 7 summarizes those program characteristics which might be 
expected to benefit the amount of gain found in each program. The RBS ' 
program may be characterized as "program" oriented, since it involved 
^arefully developed training materials and procedures. The MC program, 
in contrast, capitalizes on instructor capabilities and extended contact 
with the instructors (a relatively long training period). Finally, the 
NSD program benefits are student based in that better students (higher 
RGL and level of education) enter the program and there is a more liberal 
policy of attriting students not showing progress or students demonstrat- 
ing improper attitudes. 

The gain scores and post-test scores presented in Table 6 indicate 
that despite considerable differences in program orientation, all of thr 
programs yield approximately the same amount of gain and terminal level ^ 
of reading performance. Potentially, training of some of the men in the 
RBb program was terminated after 3 weeks due to their achieving a 7 1 RGL 
or higher on a test. However, all other men as well as the men in the 
other 3 programs were required to complete a fixed course of instruction. 
Additionally, m all programs, except perhaps the FY 1974 NSD program 
training materials went wftll beyond the sixth-grade level. Thus the' 
sunilarity between programs in the final level of 'achievement does not 
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TABLE 6. MEAN READING GRADE LEVEL* ON PRE- & POST-TESTS, 
GAIN IN READING, AND ATTRITION RATE FOR 
FOUR READING PROGRAMS. 







READING SCORES' 




ATTRITION 




PRE 


POST 








TEST 


TEST 


GAIN 




RBS (N=i?3)** 


i^.S 


6.3 


1.8 . 


.00 




3.5 


5.7 


2.2 


.10 


?m NSD(N=785) 




. 6.0 


1.8 


.22 


FY75 NSD(n=658) 


^.2 


B.l 


1-9 


.24 



READING WAS ASSESSED ON THE VOCABULARY AND COMPREHENSION SUBTESTS /' 
OF THE GATES-flAcGINITIE READING SURVEY D. 



THE "N" DOES NOT REFLECT ATTRITION FROM THE PROGRAMS 
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TABLE 7. PROGRAM CHARACTERISTICS WHICH ARE EXPECTED 
TO RESULT IN RELATIVELY GREATER REPORTED 
GAINS IN ^READING. 



PROGRAM 



1 ■ 

Characteristics 


^RBS 


MC 


FY74 
NSD 


FY75 
NSD 


Greater Instructor .Qualifications 




X 






Longer Program Duration 




X 


X 


- • 


Greater Training Materials 

CXPcNDITURE 


X 








Greater Individualization 


X 








Higher Entry Reading Levels 


X 




X 


X 


Higher Educational Levels 






X 


X 


Higher Rate of Attrition From 
Pre to Post Test 






X 


X 


Greater Statistical Regression 
From Pre to Post Test 


X 




X 
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seem to be due to a ceiling effect created by a commonality in training 
' criteria. The difference between the programs in terms of amount of gain 
is only a .4 RGL. Considering the substantial differences in character- 
istics between the programs, this small of a difference in effectiveness 
suggests the conclusion that mMiLYi tkz CLon^nz6 o£ a ^ko/vt duAajtion, 
^ui6i^dicxoh.-b(Uzd, ZoiA) tUoAacy p/wgAm, tkd omQUYit oi InatAiictLonal gain 

QznoAaZ titoAaciLj ^ {^ixzd and dzteMmimd by tkz /iAjnplz nxpo/^uAz to 
thz (Lduca;Uo)njzt ^oXZing . This conclusion is further supported by the 
1.2 RGL gain in general literacy obtained in the Army's Functional Liter- 
acy Training program (Sticht, Caylor, Fox, Hauke, Jamea, Snyder, & Kern, 
1973). This program, which . underwent considerable experimental develop- 
ment, employs qualified teachers, is highly individualized, and focuses 
on functional reading skills. Nonetheless, it possesses the defining 
characteristics for my conclusion and the gain in general literacy is 
comparable to the gains obtained in the naval service. 

Results obtained by Shennum, Aiken, and Thomas (1975) suggest that 
this invariance in reading gains may even' apply to specific instructional 
procedures. These investigators examined L:.ree procedures for increasing 
reading speed for recruits in the final week of the FY 1975 NSD program 
\rith the hypothesis that excessively slow reading speeds (frequently 
found in these programs) may be an important contributor to poor compre- 
hension. The training method of primary interest involved simultaneous 
reading and liste^.Lng with the rate controlled through a variable rate, 
speech compressor. Comparison conditions were (1) simply listening to the 
compressed speech, -. .j (2) reading with ho listening but with rate goals 
specified and feedback "given. Recruits in these conditions spent tx^o 
hours per day for six days reading a 40,000-word novel witten at the 8th~ 
grade level. A pre-test/pos t-test evaluation using different materials 
indicated that each condition produced an increase in reading rate of 
about 60 words per minute and a 20% increase in comprehension. These 
gains, while constant across conditions, were nonetheless due to train- 
ing, since a no-treatment control showed no gain in rate or comprehension 
over the six -day period. 

The general conclusion I have drawn fx.-jm" these evaluations should not 
lead to the assumption that literacy training programs cannot be improved. 
A consideration of alternative criteria for training success (the criteria 
sra currently amount of gain and exit level in general literacy), a shift 
to training of nen with .\ess deficiency, or the introduction of refresher 
or retraining programs may all yield more effective reading programs. 
Perhaps the most important consideration is a rethinking of tfie criteria 
for training success. In most Navy training, prog7^«Lm effectiveness is 
heavily weighted by the time required to reach criv.erion. The time factor 
is very important, since students and stafiE are all on salary. If a time 
criterion were applied to the reading programs, it is highly probable that 
individualized, computer-based programs employing mastery learning tech- 
niques would be by far the most cost-effective (Atkinson, 1972; Ball & 
Jamison, 1973; Bloom, 1974). The only time information available on the 
present programs ccmF* rrom RBS, where it was found that on the average, 
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41 hours of instruction was required for a one-year gain in reading* 
However, that one-year gain required 52 hours of instruction for recruits 
without decoding skills, bat only 25 hours when recruits could decode. 

The criterion for literacy training should also be rethought in terms 
of the ultimate goals of the program.. Ihe goal is not to produce laore 
literate but rather, to give them- the skills necessary to effectively 

perforra t...^ reading tasks required of them in the service. These skills 
involve an ability to find' information readily in manuals, to read and 
. craprehend procedural directions and instructions, to caterraine the rela- 
tive iinpor"tance of information on a system, etc. The vocabulary require- 
ments and the reading strategy required in these tasks differ considerably 
from the requirements in general literacy. Tests based on service-relevant 
reading material and tasks will very likely sr'jw program gains not reflect- 
ed in general literacy tests. 

Still related to the ultimate criterion of increased Nav>^ effective- 
ness, a program addressing men 'with minor reading deficiencies may be 
expected to yield greater gains in perf onrir.nce effectiveness as compared 
to Icw-literate programs even if both programs produced the same gain in 
the same time. VThile the advantages of the former program were discussed 
earlier 5- no information was available on the relationship of reading im~ 
provc^ment to entry reading level. Data derived from these literacy pro- 
grams indicate that in addition to the gains in Navy effectiveness, a 
program for moderately literate men would require less training time to 
achieve the same gains in reading. The NSD personnel refuse to accept 
less than 3.0 RGL readers as they have found training of these men to be 
'Virtually impossible. RBS reports that when a man is not proficient in 
phonic skills, he requires twice as much training to produce a year gain 
in reading. In my 6\7n v/ork at NSD, I have found a strong linear rela- 
tionship between entry level and gain (r = .35 and .59 in two samples of 
native English speakers). 

Li-teracy training programs have frequently been criticized for the 
tendency to be one-shot programs. Clearly, meaningful permanent gains in 
reading cannot be expected in a three-week pragram. Exercising of the new 
./Skills and refresher training is necessary. The NSD personnel retested 58 
graduates from the FY 1975 program two to seven weeks after graduation. 
The delay between the final test and retest did not produce any ■ systematic 
differences in change scores, which indicates that the loss due to termi- 
nation of training, occurs within two weeks. The average graduation score 
for. these 58 men was 6.3 RGL, while the average retest score was 5.1; 
a 1,2 RGL loss in reading skill due to the .tet-mination of training. If 
these men were representative of the FY 1975 input into the program, these 
data indicate a .9 RGL permanent gain from the entry reading level of 4.2 . 
IlGL- However, this gain is still* subject to some regression effect, and 
GO the permanent gain in the program is likely a .5 to RGL. While re- 
test data is not available on the other Navy programs^ there is no reason 
to suspect the permanent gain would be any greater than that obtained at NSD 
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The exit reading score indicates that the men have the capability of 
reading above the level obtaijied in retesting. To achieve that higher 
level on a permanent basis, however, will require mini-refresher courses 
over an extended time frame. 

The fact that the reading programs produce approxijnately equivalent 
gains and that there is considerable loss in skill after a man leaves the 
program should not detract from their relative effectiveness. The pre- 
test and post-test gain of approximately two years in these compulsory 
and time-compressed programs is indeed significant. Looking at the gains 
m another way, the men, on the average, ranked at the 30 percentile for 
5th-grade students at entry into training, while on the post-test, the 
men were at the 69 pei-centile. Thus, during the course, of approximately 
four weeks of training, these men move from the lower to the upper third 
of the distribution for this particular reference group. The 2 RGL gain 
is also significant relative to an average .5 RGL gain achieved in civil- 
ian adult school programs of the same total instructional hours, but 
spread over five months (and therefore, providing distributed practice) 
and offered on a voluntary basis (Kent, 1973). SiAilarly, experimental 
reading programs for school children at the same reading skill level as 
the recruits have produced less than a year gain in a year of instruction 
vBattelle Institute, 1972). In comparison, the naval service programs 
have proved exceedingly effective in improving reading skills. My dis-- 
cussion, instead, addresses the questions of whether more effective pro- 
grams can be developed and whether the programs produce gains to the 
. service beyond the increase in reading, 

V/ith regard to the effects* of the reading programs on later Navy 
effectiveness, graduates of the NSD and HC programs' have been tracked 
through boot camp to determine their rate of attrition.. The success of a 
sample of 301 MC program graduates was assessed 3 to 14 months after grad- 
uation. The attrition rate of this sample was 12% as compared to an over- 
all boot camp attrition rate of 10%. Since boot camp is only 11 weeks, 
these data indicate tha: graduates, of the MC program have an attrition' 
rate equivalent to or less than the Marine Corps average. Additionally, 
only 33/; of the graduates wt:re judged to have a promotion rate slower than 
normal, while for 17% the rate was judged to be faster than normal. Unfor- 
tunately, an untrained sample is not available for comparison. (Perhaps 
most men are typically jud3Zd to be progressing faster than normal.) The 
data do suggest that the MC program graduates are progressing through the 
service at a normal rate. However, we do not know to what degree reading 
ability is predictive of performance in the Marine Corps. (It is evident 
that the reading requirements are less than in the Navy.) Thus, the 
•'normal'* performance of these graduates may be due to some characteristic 
of the reading program or it may be due to the lack of any relationship 
between reading ability and service effectiveness. 
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Boot camp attrition rat^ for a sample of 387 graduates from the 
FY 1974 NSD program was 12%. This can be compared to an average 10% 
attrition rate at the boot camp and an attrition rate of 18.7% for 
fourth-* and sixth-grade readers. As with the MC program, the graduates 
of the NSD program appear to perform as well as the average recruit. 
Here, however, we have data pointing to a relatively strong relationship 
(between reading and attrition (see Table 2) and thus, the 12% attrition 
rate for graduates appears to be of meaningful significance. 

The graduates of both the MC and NSD r'eading programs appear to be 
performing as average recruits. However, a basic question is whether 
their level of performance in the service is due to the improved skill 
level and attitude obtained from the reading program. While a definitive 
answer to this question is not possible, it seems highly unlikely that 
1 RGL increase in ability would substantially aJEfect the performance level 
of a functionally illiterate recruit. A more likely effect of the pro-- 
gram is that they serve '^o filter out recruits having low capability for 
learning or having a poor attitude, ■ These people are represented in the • 
12% and 14% attrition rates in the MC and FY 1974 NSD programs respec- 
tively. The only way we are going to be able to adequately assess the 
effectiveness of reading training on later service effectiveness is 
through an adequate experimental design in which only a portion (randomly 
selected) of the personnel eligible for reading training are actually 
assigned to the program and the program resulted in meaningful gains in 
reading skill. A judgement of reading training effectiveness would then 
be based on attrition rates from the point of assignment, rather than 
examining on3.y the graduates of the reading program. 
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. Experimantal Reading Programs 

In addition to the ongoing programs for recruits, thefe are ^hree 
on?,fr^!^''\^''°^''r^ ""^^^ sponsorship. Since thesa programs are • 

tti V \ t . t^^^^^ °^ development, I will only briefly describe 
■deJol'o f "'^^^ Montague, under the sponsorship of the Center, are 
skills ^^''^^°f^^^°^ training phonic, or decoding, and vocabulary 
has a' U l r l\^^^-te recruit. The training is computer baseband 
Mon :f evaluation of training procedures and the evalua- 

Puter fv^riT''"'^'' hardware and softwate systems. The interactive com- 
puter system has graphic capabilities allo^^ng the student to "point" to ' 
Lm^; f ' syllabicate a word shown on the screen. More import- 

VorJL Sm.p' training, the terminals are equipped with a 

lits^ Thi^ computer-generated speech' capability per- 

Presentf tion individualized training program with computer 

m?nr?^J. ^ °^ auditory and visual mode.' A program- 

J^xng language incorporating the graphic and voice synthesizer capabilities ' 

±vl TsTeXr'' ^r:'^'^' courseware for Se phonic skLI trli" 

mg xs neanng completion. 

The second experimental program is ' being developed by Carver (1973d) 
under the sponsorship of the Office of Naval Research. The training in 
this^program. rather than focusing on specific reading skills, will involve 
150 hours of practice on the tem.inal objective - reading a^d compS! 
Heading .prose material. Prose passages will be presented over a PLATO ■ 
sZTs^^^lT ' permitting individualized instruction. The pas- 

ev!ry T±fth LlT""T. -°dified cloze (Taylor, 1957) format in which 

^^n7^ni ^ M un ^f^^^^^ "'"^^"^ '"^^^^ ^ multiple choice selec- 

tion to fill the blank. This procedure forces continued attention to the 
passage and provides constant monitoring of performance'. The primary 
objective m Carver's work is to determine if there are two types of poor 
Itt / 7- f ^ lacking the intellectual capacity, and those lacking 
t^ainJnr ^^ff 1^"^^ expected to gain from realing 

S i"^' f"^"^^"^^ "ill be screened for intellectual ability usinS 

the Raven s Progressive Matrices and high and low Raven's scorers will be 

v17r^''tt\ t ^^^^^^l"g training will consist of 4th-grade level readers. 

Pilot testing of this system has recently begun. 

Jn^-f^'"''^ experimental program involves the evaluation of the Encyclo- 
pedia Britannica program for functionally illiterate adults. The program ' 
IS at a considerably lower level than the other programs we have dLcussed • 
and assumes no reading ability. Training is in pronunciation, listening 
and basic reading skills. This progra:n has been instituted aboard thref' 
sh..p. and participation is voluntary. The experimental evaluation of this 
lilies °' ""^^ end,T„i„,„, supp„», i3 J„s^*^= 
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Prediction of Training Success 

As a final topic in- the review of the Navy's literacy research, 
I would like to present some of the results obtained in my own research 
efforts to characterize the successful student in the NSD reading pro- 
gram. For the past several months, we have been administering a battery 
of tests to all recruits entering the program and examining test scores 
in relation to the amount of gain^ in reading ability. Our first sample 
consisted of 111 recruits, 32 of whom had English as a second language, 
we then made some modifications in the test battery and have recently 
oegun collecting data on a second sample. This sample at present con- 
} recruits, 21 of whom have English as a second language. It 
Should be borne in mind that the data I am presenting applies to the 1975 
KbD program which was described previously. Programs differing in in- 
structional strategies would likely produce different relevant variables. 

One of the first hypotheses we examined was that the remediability 
oi- poor readers is dependent upon their intellectual ability. This is 
a basxc hypothesis in Carver's (1973d) experimental reading program dis- 

^° ^^^^ ^^^^ hypothesis, we used the pattern match- 
ing (PM) test (derived from the Raven's Standard Progressive Matrices as 
described previously) as a part of our battery. The split-half reliabil- 
ity (corrected for attenuation) of the PM, based on the scores of 1 200 
recruxts, is .85. Our dependent variable was the gain in reading scores 
over the course of the reading program. If a student was attrited from 
the program, he was given a score of 2.6 - .4 RGLs beloW the lowest 
exxt score. Mean PM performance was consistent across the four groups 
(successive samples crossed .^th English as a first or second language) 
ranging from 19.5 to 22.5 (a = 4.9 to 6.6). This is significantly below 
the mean of 28.3 obtained for all recruits at the San Diego boot camp. 
The mean pin in reading ranged from .8 to 1.8 RGLs, but was not consis- 

Tll ''^^^^^'^ ^° ^ language group. The gains and the correlation of gain 
and PM performance is presented in Table 8 for the English as a native 
language (E Ist) and English as a second language (E 2nd) groups. 

The correlation between PM and gain is low and not statistically sig- 
nificant (p > .05) across all four groups. If anything, there is a ten- 
dency toward a negative correlation between gain and pattern matching for 
the E 1st group. Thus, intellectual ability of low literate recruits is 
not predictive of their gain in this reading program. This result is con- 
sistent with our -finding with a random sample of all recruits, where it 
was found that PM was not strongly related to general reading ability • 

T. ^^^""^ ^^^^ whil^ intellectual ability is a component in 

reading skill, it alone does not set the limits for the level of skill 
attained m a reading program, at least as found in our heterogeneous 
adult population. 



4 

Analysis involving post-test scores have yielded comparable results. 
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J ABLE 8. MEAN (STANDARD DEVIATION) READING GAINS* & THE CORRELA- 
TION OF GAIN WITH PREDICTOR VARIABLES FOR NATIVE 
LANGUAGE (E 1st) & ENGLISH AS A SECOND LANGUAGE (E 2nd) 
GROUPS IN TWO SAMPLES. - 



CORRELATION OF GAIN WITH: 







PAHERN 


DECODING 


. MEAH 




. GROUP- (N) 


MATCHING 


SKILL 


GAIN 


SAMPLE 


E 1st (79) 


- .02 , 




.8 (1.7) 


1 


E 2nd (32) 


.19 . 


.30 


1.8 (1.9) 


SAMPLE 


E, 1st (21) 


- .39 




1.23 (1.^19) 


o 

L 


E 2nd (29) 


.18 


.28 ' 


1.0 (1.1) 



*READING GAIN IS CALCULATED AS THE POST-TEST/pRE-TEST DIFFERENCE ON 
ALTERNATIVE FORMS OF THE VOCABULARY AND COMPREHENSION SUBTESTS OF 
THE GATES MacGINITIE READING SURVEY D. 
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Our second hypothesis was that thjose students more proficient in 
decoding printed wotds into phonological units would show greater gains 
in reading. This is an often reported finding by the instructors in the 
reading program and in the NSD program it is the rationale for rejecting 
recruits below the 3.0 RGL. The word knowledge subtest of the Wide Range 
Achievement Test x^as included in our battery to test this hypothesis . 
The word knowledge test involves the presentation of a graded series of 
printed words with the subject's task being simply to pronounce the words 
correctly. Correctness of pronunciation is a test administrator judge- 
ment and grading is liberal with respect to regional and ethnic differences 
in pronunciation. The test does not cake into account the . student ' s knowl- 
edge of the meaning pf the words. 

Results obtained, with the decoding skill test supported the hypothe- 
sis. Performance on this test was consistently one of the best predictors 
of g3.±n, with the correlation being somewhat higher for the E 1st sample 
(r - ,40 and ,54) then for the E 2nd sample (r = .28 and .30), Thus, as 
indicated by the instructors, decoding skill is an important determinant 
of ability to learn to read in a short duration program. Decoding skill, 
however, is clearly not the only factor. The mean decoding performance 
(in grade levels) for the E 2nd samples was 10.1 as compared to a mean of 
4.7 for the E 1st, a difference of 2 standard deviations. The magnitude 
of this difference was consistent across samples, yet the gain in reading 
for the £ 2nd group is not consistently greater than for the E 1st group. 
We feel that a second m^jor distinction between the language groups, which 
will account for differences in gains, is semantic knowledge. The E 2nd 
consists primarily of Filipinos x^ho have been taught decoding rules for 
English since grade 1. However, their experience with the language has 
been largely limited to schoolroom use. Thus, while they can readily de- 
code the printed -x^ord, their semantic knowledge is deficient relative to 
the E 1st and will affect their progress in the remedial program. 

V7e included reading and listening tests in our battery to test the 
above hypothesis, as well as to test the hypothesis that a large differ- 
ence between reading and listening skills is predictive of the gains to 
be made in a remedial program. The tests were 'developed from the vocab- 
ulary subtest of the Nelson (1962) -reading test. • The odd--numbered items 
on the test formed our reading test and the even-numbered test items formed 
the listening test. Presentation of reading and listening items was alter- 
nated in blocks of five. "Listening'^ test is somewhat of a misnomer, since 
both reading and listening items were presented via slides at a 12-second 
rate. For the listening items, however, the stimulus and alternative were 
read to the subject while he viewed them. Split-half reliability of the 
listening test calculated on our first sample yielded an r = .79 (cor- 
rected for attenuation). 
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The raeao, listening and reading performance for each, language group 
is. presented in Table 9. The data indicate, as might be expected, that 
E 1st recruits have a larger listening, than reading vocabulary. Their 
listening skill is exercised constantly through interaction in an English 
language coinmunity, while their reading skill training- is likely to be 
limited to use in the educational system. The E'2nd, on the other hand, 
have equivalent reading and listening vocabularies indicative of an expos- 
ure to English, spoken or written, which is limited to the classroom. The 
data further indicated that the E 2nd group has a vocabulary knowledge 
greater than the reading vocabulary of the E 1st group, but less than the 
E 1st listening vocabulary. The E 2nd, while being able to decode far 
more words than the E 1st group, have more limited semantic knowledge of 
those words. 

We have calculated the correlations between reading and listening 
skills and gain scores. However, the pattern of correlations is not stable 
or clearly interpretable. Thus, I will delay discussing that data until 
our second sample is complete. We have included a number of additional 
measures in our battery for the second sample. These include some of the 
traditional measures of Navy effectiveness, such as number of dependents' 
and years of education, as well as tests of immediate memory and ability 
to maintain a high level of performance at a tedious task. The intent 
of these tasks is to look for what may be general ability deficits in our 
low reading ability recruits. 



TABLE 9. MEAN (STANDARD DEVIATION) READING AND LISTENING 
ABILITY SCORES FOR NATIVE LANGUAGE (E 1st) AND 
ENGLISH AS A SECOND LANGUAGE (E 2nd) GROUPS 
COLLAPSED ACROSS TWO SAMPLES. 





Reading 


Listening 


E 1st 


26.1 
(5.1) 


31.1 
(5.0) 


i: 2nd 


29.7 
(5.0) 


. 28.0 
(^.8) 
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COMMENTS ON THE PAPER BY DUFFY 



Burl Gray 
Behavioral Sciences Institute 

The comments by Gray focused on the parallels between the concern 
shown in Duffy's paper for understanding the characteristics of the 
reader, and those same kinds of concerns so frequently found in the 
civilian education world. Gray pointed out that he thought too much 
attention was given to understanding the learner, and not enough atten- 
tion was given to describing the instructional method that students are 
exposed to. He suggested that the only way one will be able to accurately 
estimate reading gain as a function of time is to have a teaching method 
highly specific to some goals or objectives; then one can make a meaning- 
ful statement about gains per hour of instruction. In addition to making 
predictions of success in a program more accurate and meaningful, having 
a clearly defined method and procedure is useful when investigating fail- 
ures from that program. In this case, one can make definite adjustments 
to the program until the failure rate is minimal. Gray closed his remarks 
by reiterating that in the military work, more attention needed to be 
given to specifying instructional methods and procedures. Continued worry 
about the characteristics of the reader may imply that the fact that he 
does not read is problem, so trainers may not be too motivated to 
improve; tkzAJi work, and the problem iliay be expeditiously solved by dis- 
charging the person or changing his job assignment, or some other change 
in category rather than competency. 
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ONGOING R&D IN ARMY LITER^^CY TRAINING 

John S, Caylor 
Human Resources Research Organization 

The Army's problem in dealing with recruits of low level reading and 
language skills is a recurrent one. In the past, the response to this 
problem has typically been the initiation of brief, limited, stop-gap 
programs of general literacy training whose content and objectives bore 
little demonstrated relationship, to . the requirements of Army job training 
and job performance. The most recent recurrence of this problem arose in 
1966 with Project 100,000, which brought large number^j of previously 
ineligible low aptitude men into the Army. At this time, the Army under-- 
took a program of research and development which was completed in 1975i 
Tnis paper describes that program of literacy research and development 
activities, and briefly mentions the now ongoing research. 

As an immediate response to the influx of ' marginally literate person- 
nel^ under Project 100,000, the Army established Army Preparatory Trait? ing 
(APT), The goal of this program was to bring recruits reading below the 
fifth-gtade level on a standardized reading test to that criterion before 
their entry into basic training. With widely varying local options, this 
program provided general educational development training in reading and 
arithmetic for a period of not more than six weeks, at which time all 
students were advanced to Basic Combat Training regardless of their read- 
ing achievement, - 

Shortly after the APT program was established, the Army, under .con- 
tract with HumRRO,' undertook a systematic long-range program of literacy 
research and development projects which had two major objectives: 

1. To determine and define the nature of literacy problems in the 
Army by studying the actual operational literacy demands of 
Army jobs in conjunction with the literacy skills of Army person- 
nel. 

2. To develop a literacy training program to provide job-demanded 
functional literacy tikiJ Is to meet the reading requirements of 
Army jobs. 

Detailed descriptions of the projects undertaken to meet these goals 
have been reported elsewhere (Sticht, 1975c). The earlier phases of this 
program. were primarily of a research nature and these will be summarized 
in terms of the work to determine the functional reading requirements of 
Army jobs and the reading ability level of the Army personnel available 
to do these jobs. The latter phases of this program were more heavily 
developmental; those phases will be described in terms of the job func- 
tional literacy training program which was produced, its operational 
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effectiveness under' Anay-wide implementation as a front-loaded program . 
conducted before job training,^ and the feasibility of integrating this 
''job reading training program with job skills training. 

DETERMINATION OF THE LITERACY DEMANDS OF JOBS 

To determine the literacy requirements of Army jobs, three different 
approaches were used. In these three approaches, the reading requirement 
of a job was established in terms of one of the following: • 

1- Direct measures of job knowledge arid job performance. 

2. Tlie readability (reading difficulty le^vel) of the Army manuals 
prescribed for use in learning and in doing the job. 

3. The specific job reading tasks inherent in performing the job. 
Each of these approaches is described below. 

Job Proficiency 

Our first approach to determining the literacy demands of jobs was 
to examine the relationship between the general reading ability level -of 
job incumbents and their job proficiency. Job proficiency was measured 
both by a written test of job knowledge and by an extensive hands-on 
sample of job performance. For each of four Military Occupational 
Specialities (MOSs) , the literacy requirement of the job was estimated as 
the lowest reading grade level at which no more than a chance proportion 
of men fell in the bottom quartile on the job proficiency measures. 
These analyses indicated a consistent relationship between literacy and 
job proficiency, and suggested the requirement of 7th-grade reading level 
for Coqks, 3th-grade reading level minimal requirement for Armor Crewmen 
and Vehicle Repairmen, and 9tTi-grade reading level for the Supply Clerk's 
job. 

This approach to establishing the reading requirements of jobs has 
the advantage of using job proficiency measures directly as criteria. 
Tliis work suggested th<?X (different jobs do have different levels of lit- 
eracy demands and tUat reading requirements for these four high-density 
jobs ]f2.y well above th-'i Iv^vels typically set as obj ectives f or remedial 
reading training prograci-;. There are drawbacks to this appraoch. Clearly 
it is prohibitively expea;ive to obtain hands-on job proficiency measures 
in a variety of jobs. A different problem arises from the job proficiency 
measures themselves, for they represent the resultant effect of many fac- 
tors, of which literacy is only one. Since the job proficiency approach 
totaIl;y 'gnores the job reading materials themselves, we then turned to 
the study of those reading materials used in training for and performing 
the job. 
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Readability Approach . 

Cur second approach to determining job reading requirements was to 
study the readability of reading difficulty level of Array manuals used 
on various jobs. The FORCAST readability index was constructed to esti- 
mate the reading grade level of ability needed by the.^adult Army popula- 
tion to read and comprehend technical job reading materials (Caylor, et 
al,, 1973). FORCAST estimates of the readability of manuals indicated 
these results: 

1. More than half of the job manuals in each of seven jobs 
exceeded the llth-grade level of reading difficulty. 

2, The average readability level, of the materials in each of 
these jobs far exceeded the average reading ability of men 
working in these jobs. 

Table 1 shows that although there are clear differences in readability 
of job printed materials among the MOSs, all the MOSs show readability 
levels well above the 9th-grade level, 



TABLE- 1 

Cumulative Percentage Distribution of Job Reading Materials for 
Seven MOSs and Seven FORCAST Readability Levels 





Military Occupational Specialty^ 




11820 
















26D20 


63B20 


71H20 


76Y20 


91B20 


95B20 




(N = 104) 


(N «95) 


(N - 108) 


(N^SS) 


(N^SS) 


(N =» 90) 


(N^-ISS) 


6-6.9 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


7-7.9 


1.0 


0.0 


0.0 


Q.O 


CO 


0,0 


0.7 


S.S.9 


4.8 


0.0 


3.3 


t.1 


3.5 


2.2 


5.0 


9-9.9 


18.3 


4.2 


13.4 


2.2 


10.8 


24.4 


15.1 


• 0-10.9 


41.4 


9,5 


35.3 


3,3 


20.4 


47.8 


34.0 


11-11.9 


71.2 


42.1 


61.8 


37.0 


57.6 


77.8 


62.2 


12.CH- 


100.0 


100.0 


100.0 


100.0 


1000 


100.0 


100.0 



^RGL~Reading Grade Leval of difficulty of job printed materials determined by the FORCAST 
formula. 



"11B20, Light Weapons Infantryman; 26O20, Ground Control Radar Repalfman; 63820, Wheel 
Vi?hi3)>» Repoirm,-:n; 7 J H20, Personnel Spocialist; 76Y20, Armorer/Unit Supply Spedalist; 91 B20, Medical 
Specu-)!i3tr 95320, Militar/ Policeman. . . ■ 

The readability technique offers a low cost method for estimating the 
overall reading demands of job manuals. However, it does not provide a 
direct indication of how well men can read and use their manuals for the 
veading tasks performed on the job; for this information, we need to test 
people on samples of job reading tasks using job reading materials. 
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Job Reading Task ■ ' 

Our third general approach to assessing job literacy requirements con- 
sisted of studying directly the relationship between general literacy 
skill and performance on job reading task tests; that is,- reading tests 
constructed of actual job reading materials used in performing actual job 
reading tasks. Job > reading tasks were identified by means of structured 
interviews with job performers at their work location. Job performers 
were asked to report instances of their use of printed materials in per- 
forming job tasks, to describe the information they sought to perform 
the job, to obtain the printed material, and to indicate the needed job 
infor-mation la the manual. These verified .job reading tasks were then 
structured into Job Reading Task Tests which were standardized and normed 
on Army samples whose general reading ability level was also measured. 

These tests consist of the most frequently mentioned types of reading 
material and require the testee to obcain the .same kinds of information- 
.from the same manuals as job incumbents reported using in their work. 
Thus, they represent the most direct measure of actual job-specific read- 
ing task performance. 

Each of the Job Reading Task Tests (JRTTs) constitutes a set of con- 
tent-valid, job-specific reading tasks which can be used as a criterion- 
referenced measure of job reading task performance for that job. To facil- 
itate comparison of job reading requirements, as defined by a JRTT, ' 
between- different jobs and with other indices of job reading requirements, 
JRTT scores were scaled in terns of reading grade level, as measured by a 
standardized reading test. Thus, any job-specific JRTT score could be 
expressed as the performance of a soldier whose general reading compre- 
hension was at some specified reading grade level, to use that common met-- 
.ric. In this fashion, and using the' arbitrary but plausible decision 
rule that 7C^ of job incumbents should get 70% of the items correct on 
their JRTT, the reading requirement for Cooks fell at about the 7th-grade 
level, for Vehicle Repairmen pt the 8th-grade level, and at tho. 12th- 
grade level for Supply Clerks. 



Summary 



These were our three main approaches to determining the reading level 
requirements of jobs. In each, we studied the relationship of general 
reading ability to a different criterion: measures of job proficiency, 
the structural properties of job reading materials, and the performance 
of empirically determined job reading ta^ks. These approaches agree in 
general in estimating substantially different reading requirements for 
different jobs, in a commoi. orderi:^g of the literacy requirements of the 
three jobs studies for all approaches, and in setting all job reading 
requirements at or above the 7th-grade reading level. 
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up to this po\-it, we have -discussed research that has focused on the 
reading demands of rry jobs. The other side of the Army's literacy 
problen concerns th*i reading ability level of the personnel available to 
dc the jobs. 

Direct assessment of the current distribution of reading ability in 
Amy is not available because no reading test is standardly adminis- 
tered to all Army personnel. Throughout the several phases' of this 
program, direct ' testing of reading comprehension has always yielded a sub- 
stantial proportion of personnel' of marginal reading ability. While this 
proportion fluctuates as a function of the quality of Army input, it is 
not likely to become trivially small. Most recent data as' of March 1975 
• show that of 23,000 Mental Category III recruits screened on a standard- 
ized reading test, 17% failed to reach the 7th-grade reading level, and, 
of the. 13,000 Category IV recruits screened, 43% were similarly reading 
at the 6th-grade level or lower.. Subsequent retesting five weeks later 
xn basxc training reduced these percentages by half - which still leaves 
some 13% of Category III and IV input reading no higher than 6th-grade 
level at the time when they are about to enter job training. 

An alternative to the direct assessment of the reading ability of 
Army personnel is provided in our study by the comfortingly consistent 
correlation (r = 0.7) between AFQT and reading grade level as measured by 
standardized reading tests. Since AFQT -cores are available for all 
military personnel, the reading skill level of personnel in any job cate- 
gory can be reasonably estimated at any time from existing data banks. 

Summary 

To briefly summarize what has been learned about the Army's literacy 
problem: We have seen that (a) as determined by a variety of methods, the 
reading demands of Army jobs,- even the less complex ones, far exceed the 
reading ability levels of many personnel, and (b) there : is a consistent 
positive relationship between reading ability and j;..b proficiency. 

From chis work we make the following conclucrlo:^.: Although no single 
level of functional literacy can adequately represent the reading require- 
ment of the range of MOSs studied, there appears to be a lower limit "of 
7th-grade reading level for functional literacy in the Army. Thus, 
remedial reading should be aimed at producing no less than 7th-grade read- 
ing ability and, optimally, should" be targeted to che level of a man's 
MOS assignment. If eliB^ination, retention, or promotion of career person- 
nel is largely contingent upon paper-and-pencil job kaowledge tests, 
formal procedures should be implemented to ensure that men have the 
opportunity to acquire both the job knowledge and the literacy skills 
required by the tests. „ • 
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JOB FUNCTIONAL LITERACY TRAINING PROGRAM 

Based oa this research or. the nature of its literacy problem, the 
Army in 1971 sponsored the HumRRO FLIT (Functional Literacy) project with 
the objective of developing a literacy training program designed to pro- 
vide a level of functional literacy appropriate to pre3t:nt minimal job 
reading requirements and requiring no more than six weeks of training^ 
time. Given the absolute constraint of six weeks of trainijng, we could 
see no reasonable prospects of increasing our adult students' general 
literacy competenc > the point where it would transfer significantly 
to his job reading sks. Accordingly, the FLIT objective was specified 
. to be that of prodi: " a .student capable of using , his job reading mate-" 
rials with the off , .yeness of a man having a general reading aoility 
of grade 7,0 ov .: .-.ht-:. as indexed by performance on a job reading task 
test. Thiii 1 .[ :a.. :asic policy decision that all job reading train- 
ing would be c using the concepts, content, and reading mate- 
rials of the d own job area. Parallel training curricula and 
materials were developed for each of six job clusters: Cook, Clerk, 
Communications, Combat, Mechanic, and Medic. 

Entrv 

" r 

Entry to the program is governed by a series of screenings designed 
to eliminate most cases of t:c:U;ing artifact. All Mental Category III and 
iV input personnel. are screened on a standardized reading test in the 
RecepLion Station during recruit processing. Those failing to reach the 
6th-gradG reading level are rescreened five weeks later in basic training. 
Those again souring below the 6th-grade reading leyel are screened once 
mo^fe upon entering the reading training at the end of basic training by 
means of both a general and a job reading test. Only those students who 
have failed all. three screenings are admitted to the job reading training 
program. ... 

Program Overview 

The functional literacy training program consists of three curricu- 
lum strmds, each of which occupies about one-third of each training day. 
Strand I was designed to provide training in the application of existing 
general reading skills to job-specific Army job reading tasks. Strand II 
was designed to improve basic reading skills and job knowledge through 
using s:Lmplified versions of Ari:;y jc^b reading materials. Strand III is 
devoted . to general reading training and practice, as prescribed by the 
local Army school. 

Strand I 

Strand I training is designed to give the student drill and practice 

in applying his existing reading skills to .the job reading tasks and the 

job reading ijiaterials which he will encounter in his entry level job 
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training and job performance. Tliis Strand is a modular, self-paced, mast- 
ery^based program of job reading task training. Training is conducted in 
six modules, each addressed to one of the six fundamental job reading 
tasks identified- as common and essential to MOS training and performance. 
These nodules provide training in six job reading tasks: Using a Table of 
Contents, Using an Index, Using Tables and Graphs, Using the Body^of the 
Manual, Following Procedu-^al Direction, and Following Instruction in 
'Filling Out Forms. Job-specific training is provided in using those 
Army manuals used in the student's own MOS cluster. 

Entry to each module is determined by a module pre-test. Students ^ 
meeting both time and accuracy criteria on a module pre-test advance to 
the next modul.*; those failing the pre-test enter successive blocks of 
individualized instruction until they can pass a module post-test. 

Strand II " 

In contrast. Strand xs a teacher-oriented program designed both 
to improve, basic reading skills by language instruction and direct read- 
ing training, and to develop baciic job knowledge in the student's own MOS 
field. In ^.his curriculum, instruction is provided in decoding print to 
speech, in basic functional ^rammar and syntax, and in conceptualizing 
and structuring the meaning of a passage in linguistic, schematic, and 
pictorial representations. Ml instruction is keyed to sets of specially 
written, simplified reading passages presenting basic job vocabulary and 
job concepts in the student's job field. 

IMPLEMENTATION 

Alter a period of development and field trial in which all training 
was conducted by research personnel, this functional literacy training 
program has been implemented ov^' the past year at a;ii^six Army Training 
Centers. The final phase of. our R&D program consisted of evaluation of 
(1) the front-loaded functional reading training program in its opera- 
tional satting, and (2) the feasibility of alternative delivery systems 
permitting the integration of the job reading training program with job .. 
skills training. 

Let me summarize the findings of this study of the operational train- 
ing program. 

1. The Current Heed For Literacy Training in the Army 

In reading screening testing for this program in 1974 and 1975, 9% 
of Category III and 21% of Category IV recruits failed to reach the 6th- 
grade reading level on either of two administrations of a standardized 
reading test administered over the first Jix weeks of their Army service. 
Similarly, one-fourth of all input to a sample of entry level job train- 
ing courses for drivers, mechanics, and supply clerks failed to reach the 
6th-grade level of reading compreheflsion. 
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2- Chirac tcristics oE Students Entering Readiag Training 

Regional differences were substantial. Median amount of fonnal school- 
ing ranged fr6a 10 to 12 years, 29% to 57% of the students held a high 
school diploma or GED certificate, and 66% to 97% spoke English as their 
priinar:; language. 

3. Program Effectiveness . 

Pre- and post-module proficiency testing showed improved performance 
on all modules with substantial differences in the difficulty and effective- 
ness of various modules. 

SunHT^ative evaluation showed high consistency of training effective- 
ness at the several installations. As measured by the Job Reading Task 
Testy- T2ean reading grade level gain was 2.2 years, with 45% of the students 
reaching the 7th-grade reading level objective. On a standardized reading 
test measure, average gain was 1.0 reading grade level, with, 29% of the 
students reaching the 7th-grade level. Re^iention testing eight weeks 
af L-:er* coTupletion of reading training showed that, ^or job r:;ading, an 
initial eain of 2.4 RGL was reduced to n net gain of 1.9 RGL two months 
j.3t:er. For the general literacy meas a. the initial gain of 1.0 years 
was reduced to a net gain of 0.4 of ft /c^ar. 

• Fea sibility of Integrating Job Reading and Job Skills Training 

A3 i Implemented, this literacy training program js front-loaded and 
acids six weeks to the training cycle when inserted between Basic Combat 
Training and entry level MOS Training. Not only is this costly in terms 
of trai.ning time, but it also requires the learning of job concepts before 
the referents of those concepts have been experienced. Accordingly, 
studies were run on the feasibility of integrating job reading training 
and job skills training in one common program. When offered as either a 
part of, or an extension to, the regular job skilj.s training day, the FLIT 
Strand I program obtained results directly comparable to those obtained' 
in the front-loaded program, despite the curtailed time devoted to the 
integrated Strand I training. As in the case of the basic FLIT program, 
this integrated job reading training was a scheduled mandatory training 
activity for poor readers. When the FLIT Strand I program was offered on 
a voluntary unscheduled, walk-in ba:iis at the post learning center, there 
were no takers. This lack, of volunatry participation in the program may 
reflect the brief period of time the program was offered and the limited 
advertisement of the program. Future programs should expect to spend 
more effort- in disseminating information about the program. 

Summary 

Tlie Army-x^ide implementation of the front-loaded FLIT program, or 
the Advanced Individual Training Preparatory Training (AITPT) as it is 
now knoxm, has just been completed in 1975. This implementation effort 
brings to a close this Ariiiy/HumRRO long-term program of literacy research;^ 
and development. 
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CONTINUirJG ACTIVITIES . ' . 

Currently, Arnry research and development in' the literacy area is 
being conducted by the Army Research Institute, Ongoing is the develop- 
ment of a screening test for reading comprehension to replace the pres- 
ently used USAFI test. Since the USAFI test is a military reprinting of 
a standardized reading test developed for use in the intermediate grades 
of the. public school system, it requires academic knowledge and skills 
unrelated to Army reading needs. ARI is currently collecting pilot data 
on its nev7 test of literal comprehension of Army-related reading mate- 
rials. 

Under consideration is a more extensive program of R&D aimed at the 
improvement of Army reading materials. The following initial major 
research activities are being considered: 

1. The definition of relevant language variables and the develop-* 
ment of computer programs to measure these variables. 

2. The determination of those writing characteristics that need 
to be revised to substantially improve readability. 

3'. Ihe development of computer programs to provide corrective 
feedback to writers. 
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COMMENTS ON THE PAPER BY CAYLOR ■ - 

Richard Venezky 
' University of Wisconsin 

Like others, Venezky based his coiments not only onCaylor's paper, 
but also on other presentations and discussions. As preliminary observa- 
tions,, Venezky noted that he had gained a lot of respect from this Confer- 
ence for the level of work going on in the Armed Services, and was sur- 
prised to discover that most of the military researchers were facing the 
same kinds of problems as those working in civilian settings with children. 
He noted that a fundamental difference between civilian- and military-based 
researchers was that the * latter are almost forced to translate research 
into practice— to show results. University-based researchers, on the 
other hand, are rewarded not so much for applying what they learn to the 
solving of problems, but for publishing research and developing terminology 
which may in fact obfuscate how their research might be turned into practice 

Focusing -pn literacy problems within the Services, Venezky stressed 
four areas of concern- First, he expressed the idea that it is not liter- 
acy alone, but the hoiomunxcation of information x/iiich should be of concern 
to the Services. He noted that Caylor had repprted a large influx of men 
with low levels of language, not just reading, skills; also, the Air Force 
is sponsoring work on the measurement of auding and reading skills. Tiiis 
suggests that the military's problem of communication extends to both 
listening and reading, not reading alone. Furthermore, the concern with 
readability research suggests that the Services are aware of writing as a 
problem. Hence, there is evidence that the Services must consider a total 
communications approach to their problems, and not focus just on reading. 

From the communications xperspective, Venezky suggested that, given a 
constant message one has to communicate, three factors are involved: the 
medium of communication (audio, visual); the arrangement and structure of 
the message (readability factor) ; and the competency of the receiver 
(language, literacy skills) . Because research is currently going on in 
all of these areas in the Services, Venezky suggested that better coordi- 
nation of these efforts is needed, because, for exaraple^^ it is simply not 
tost beneficial to deal only with, say, literacy pro.grams, when in fact 
the materials that people are expected to read are not very effective. 
Also, needs may change over time as costs change for print versus audio. 
Thus, there needs to be a coordinated research program looking at the 
total communications picture to better choose media for communications, 
to improve the structure and arrangement of messages, and to develop the 
required competencies of personnel. . 

A second, . related area of concern expressed by Venezky was the need 
for a. communications scale for various types of communications skills 
(literacy, writing, reading, speaking) and, in particular, a scale which 
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gets away from the use of grade scores. He felt that the latter scores 
are arbitrary, and in the educational setting of the military, are 
derogatory to the personnel • Rather than insulting personnel with such 
grade scores, Venezky suggested that the Services should talk in terms 
of the competencies necessary > for the jobs to be done. 

The third area of concern which Venezky discussed dealt with oral 
language. Many persons of low oral language skills, including many non- 
native speakers of English, are entering the Army, and it is necessary to 
teach them to comprehend the oral language, as well as the written 
language. With native English language speakers of low oral language 
skills, it seems necessary to improve these skills 5.f their reading is to. 
be much improved. Again, Venezky noted that the An.r Force research on 
auding and reading indicated current awareness of nhis problem. He 
suggested that the services extend this concern with oral language, and 
focus not on teaching^the grammar of English directly (the past perfect, 
the negative passive transformations, etc.) but rather, that a functional 
point of view be taken in which the functions of the language are stressed 
and trained. 

Venezky 's final comments concerned the need for the Military Services 
to explore alternatives to current training (literacy and technical) 
programs. Noting that the Services face the problem of limited time and 
funds for training, as well as high turnover of instructors, Venezky 
suggested the wider exploration of computer-managed or imparted self-study 
programs. 
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SUMMARY OF ONGOING RESEARCH AND DEVELOPMENT 
■ IN LITERACY -TRAINING IN THE AIR FORCE 

Steven D. Groff 
Air Force Human Resources Laboratory 

Before describing the current efforts in the Air Force's R&D program 
in the area" of literacy training, it would be useful to note how the 
present training program is conducted and to gain some perspective on its 
effectiveness* All recruits are administered a locally developed fifteen 
minute reading test (designated RJS-1) upon ^ their arrival at Lackland for 
basic military training/ Those scoring below the sixth-grade level and 
all Mental Category IV personnel (those scoring below the 31st percentile 
on the Airmen Qualifying Examination), regardless of their performance on 
the RJS-1, are then aciminist^-red the California Pveading Achievement Test 
(CRAT) . Those scoring below the sixth-grade level on the CRAT are^ 
assigned to the Reading Piroiiciency Unit for a period of up to eight weeks. 
In the proficiency unit the airmen's time is split between half a day of 
readinc training and half a day of the usual basic military training. 

Tile first week ±3 spent in a conventional classroom setting working 
on work power and phonics. At the end of the" week a diagnostic. test is 
administered and the results used to guide the person's progress through 
the remaining seven weeks of the program. This latter portion of the 
course is self-paced and relies primarily on the SRA (Science Research 
Associates) Reading Series material, though other materials, such as 
McGraw-Hill and Readers* Digest, are available. Placement is automatically 
at the fifth-grade level and everyone is required to progress through the 
seventh-grade level materials before being eligible for early release from 
the Reading Proficiency Unit. Having achieved that point, tl students 
are given the alternate form of the California Reading Achiev. .lent Test, 
and, if they score. at the sixth-grade lavel or higher, they are channeled 
back into full-time basic military training. If they do not achieve that 
cr-Iterioir level, they remain in the proficiency unit and are provided 
T/ich additional . training. At the end of the eight weeks, two courses of 
ac^.lv>r: are open for those who do not achieve the sixth-grade :?evel. Either 
they are separated as having "limited potential" or, if the specific 
situation warrants, they are maintained in the unit for two additional 
weeks. This latter course is at the discretion of the unit commander and 
is used in roughly three percent of the cases. 

Of the 439 entrants into the program in FY 74, 103 (23.5%) were 
discharged for reading problems, and r.r. additional 63 (15.5%) were 
discharged for "other reasons." (The general attrition rate for basic 
military training ir. FY 74 was 7.7%.) The average length of time a 
recruit stayed in t4ie Reading Proficiency Unit was 23 days, and the 
average grade level improvement was 1.8 (Nath, 1975). 
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Once out of basic training the airmen proceed either to technical 
school for anywhere from one to thirty veeks of training, or to a direct 
duty assignment, in 'terms of the reading improvement program, assignment 
to technical* school merely delays the next step of the process. When a 
man with an AQE of less than 60 arrives at his first permanent duty 
station,, he is given whatever reading tesL is locally available. If he 
scores below the ninth-grade reading level, he is automatically enrolled 
in the reading program offered by the base education office, and supposedly, 
continues until he achieves a ninth-grade reading level. Even though a 
person may initially test above this cricerion, he may be required to 
enroll in the reading program if he has trouble completing upgrade train- 
ing and/or. his supervisor somehow dotermines that his problem is one of 
reading, 'These base-level (local) programs vary rather 'dramatically 
since the requirement is only for their existence. As a result, programs 
are established on the basis of what is available in the community; this 
may be a transplanted public high schooi/GED course or it may be a course 
provided through a special contractual agreement with a university or 
other institution* 

There is no easy means of tracking all the existing reading programs, 
but it seems reasonable to make the generalization that the typical 
approach has a very traditional orientation. Such a program i- conducted 
at Lowry AFB, Colorado, through an arrangement with the Denver Public 
School System. The program is essentially indefinite in length since any- 
one who desires to attend the twice-a-week class meetings may do so for 
as long as he likes. Roughly an hour of the class is spent in lecture/ 
participation format covering phonics, work attack skills, and the like. 
During the second hour the students work with SUA materials and receive 
individual tutoring as required. Periodically, testing is done to 
A^Yl{^on^}aLtiJ track progress. but no permanent records are maintained. 

The extent to which these programs attempt innovation is quite limited. 
The Education Services Office at Kirkland AFB, (Mr. Illes, personnel 
communication), for example, has instituted an eight-week program under 
contract with the University of New Mexico. Of twelve hours of instruction 
per week, L-Gur are in the area of study skills, involving the use of 
dictionaries and orientation trips to libraries. Two hours are spent in 
group counseling sessions (a rather unique feature) and the remainder of 
the time is spent in other reading a^uivities using Air Force Career 
Development Course materials. During the first six weeks, heavy emphasis 
is placed on phonics because of an apparent need for such training. The 
final two weeks are spent building on this background by having the group 
read aloud plays which are then discussed and which serve as a vehicle for 
further skill development. 

IS THERE AN AIR FORCE/LITERAOY -PROBLEM? ^ 

Concern has been expressed at the 1972 and 1973 World Wide On-the-Job 
Training Conferences by supervisors and training managers that reading 
problems are proving detrimental to the conduct of training. Their ' 
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ppsition appears to be supported by some recent research. As a result of 
a^survey by Mockovak (i974a). it was fouad that 5,774 airmen participated 
in base-level reading programs betvaen 1 April 1972 and 1 April 1973 and 
that the most frequently cited reason for enrollment was the inability 
to read and/or comprehend Career Development Course material (that is, 
those self-study materials required for skill upgrading and promotion) . 
In addition, base education officers reported that, on the average, fifty 
percent of those enrolled in their reading programs were high school 
graduates, and that eighty-four percent of those enrolled represented only 
ten career fields. Thus, there is the further indica^iion that the more 
highly educated people within the Air Force manpower pool are not all 
adequate readers. In absolute numbers ^ the existence of a "reading prob- 
lem" appears co be. a legitimate concern, particularly if ont:^ realizes that 
the tumo\.er personnel is -a long-term process and that enlistment 
criteria arc likely to fluctuate downward from the January 1975 level, 

Mockovak (1974b) studied the literacy skills of AF personnel and the 
reading requireuients imposed by AF training and job materials. Mockovak 
estimated the readinp. ability of three- and five-skill ' level personnel 
using the Madden and Tai'es (1966) regression equation for AQE (Airman 
Qualifying Examinatio scores, and also estimatf^d reading requirements 
using the FORCAST (Caylur, et al . , 1973) readability measuue. Icientif ying 
-the reading' requirement level as thnt Reading Grade Level (RGL) below 
which seventy-five percent of the material fell, he compared this require- 
ment level to the mean RGL of AF personnel by career ladder. This calcu- 
lation resultea in a "literacy gap." Of the 56 ladders he studied, 
Mockovak found forty- three to have a negative gap (i-e,, a reading require- 
ment of materials exceeding the mean reading ability of those who were 
expected to use them). Of these forty-three, twenty-nine had a negative 
literacy gap greater than one, seventeen greater than cwo, and four 
greater than three; thus, thirty percent of Mockovak* s sample had a nega- 
tive literacy gap of greater than two reading grade levels. These results 
tend to confirm some of the increasingly frequent comments by training 
managers that reading problems are having a detrimental impact on the 
on-the-job training program. However, it is useful to norc- that these 
data do not indicate that a specific problem exists or what the operational 
consequences of the subjectively defined "literacy gap" are. 

Other data which provide an important perspective-, on the scope of the 
problem, and suggest' that it may be underestimated, were compiled by 
DeGuelle (1974), He compared performance of airmen scoring at the ninth- 
grade (the ceiling) on the USAFI III reading test with their performance 
on the SRA'Diagnostic Reading Test, Sur^'ey Section, Upper Level, (grades 
seven through college freshman year) Form A, (Within some AT^ commands 
the USAFI is widely used) , Using ninth-grade norms on the vocabulary 
portion of the SRA, 24% of the testees were identified as having reading 
deficiencies; on the total comprehension test, 227. were identified as 
deficient,. Using twelfth-grade norms, 63% of those tested were deficient 
in vocabulary, and 70% in total comprehsnsion, DeGuelle^s opefational 
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definition of "deficiency" on the SRA was the achievement of scores at or . 
lower than the 30th percentile. 

This suggests that estimates of the number of persons, with generally 
inadequate reading skills are likely to be low. To what extent the same 
can be said of job specific reading skills is unknown- 

A final consideration before progressing to the current AF R&D in 
literacy training is the recent change in enlistment standards which 
will surely have some effect on AF reading training, particularly on the 
operation of the Reading Proficiency Unit described earlier. In January 
1975 the enlistment standard was raised to a General Aptitude Index (GAI) 
score of 45 with a composite of 170. Shortly thereafter the Air Training 
Command (ATC) initiated a 90-day data collection effort at Basic Military 
Training at Lackland AFB, A for ty-percent random sample of recruits was 
administered the California Reading Achievement Test, A total 'of 5154. 
non-prior service personnel were tested. Forty-three percent of those 
tested scored at the thirteenth-grade level and only 3,7% scored below 
the ninth grade. (The officially recognized standard for reading ability 
is the ninth grade.) The mean reading grade level was 12,1 and the 
st^^adard deviation was 1.6. Assuming this enlistment standard is main- 
tained, no one with a GAI less than 45 should be entering the system, the 
expected result being that there will be essentially no one below the 
ninth-grade standards. This is an important consideration in planning 
the direction of AF research. At the same time, though, historical 
trends suggest that entrance standards are likely to fluctuate in the 
future (Dailey, 1951; Lecznar, 1962; Vitola, Valentine & Tupes, 1967); 
with the current high standards, that fluctuation will be downward. 

CURRENT LITERACY R&D IN THE AIR FORCE 

Research and development in the AF has only recently begun to con- • 
sider in a systematic fashion the area of literacy training and two major 
efforts are currently ongoing. The first of these calls for the develop- 
ment of a literacy assessment^ battery and is based on Sticht's auding- . 
reading model (Sticht, et al, , 1974), 

The rationale for the literacy assessment battery lies • in the basic 
processes involved in the acquisition of language. As a person moves from 
an initially pre-linguistic stage and interacts with his environment, he 
begins to develop auding and speaking skills. These aspects of languaging 
continue to develop as the person enlarges his conceptual base and begins 
to make associations between written and spoken language. As the person 
progresses through the- early years of school, he begins to acquire those 
skills involved in reading and writing; up until about the eighth grade 
(that is, through stage 3 of Sticht 's model) a person's oracy skills 
generally surpass the "literacy" skills of reading, and writing,^ At about 
that point the person has generally acquired the prerequisite^ decoding 
skills which permit the oracy-literacy gap to narrow, after v;hich the 
skills then develop simultaneously. 
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167 



The extent to which cognitive content and processes have developed 
is a constraint upon the acquisition of further oracy and literacy 
skills. Also, within this framework, the development of literacy skills 
is limited at any given time by the current level of oracy skills as 
well as the degree of mastery of decoding skills- The corollary to this 
is that, practically, one can only hope to improve literacy skills up to 
the level to which oracy skills have been developed. 

It is upon this proposition that the Literacy Skills Assessment- 
Battery is being developed. It is intended primarily for the screening 
of persons with a reading level of eighth grade and below, and currently 
is composed of three sections:, paragraph comprehension (with auding and 
reading components), vocabulary, and tracking. The first section will 
yield a /^potential" scpre indicating the discrepancy between the per.^on's 
auding and reading skills. The latter two sections fulfill an essen- 
tially diagnostic function (for decoding, vocabulary, and automaticity) 
for problems that surface in the first section. The Literacy Ass.es.sment 
Battery is being designed to overcome- the shortcomings of currently 
available types of reading potential tests (e.g.-, the Diagnostic Reading 
Scales and the Durrell Listening-Reading Series) and its advantages over 
these ter;i:<3 are that 1) ll is adult oriented, with the passages drawing 
upon experiences familiar to young adults; 2) the questions are passage 
dependent and open ended: and 3) the vocabulary section is keyed to the 
comprehension passages. 

In light of the previously noted inadecjuacy of some general reading 
tests, it is hoped that this new Literacy Assessment Battery will prove 
useful in conventional reading duvelopment programs, as well as in the 
job-oriented type program, which constitutes a second area of current 
Air Force research and development. 

In view of Mockovak's literacy gap data and apparent problems with 
the Career Development Course materials, the Air Force Human Resources 
Laboratory (through a contract with the Human Resources Research Organi- 
zation, HumRRO) is now developing a prototype job-oriented reading 
program for the Air Force. The approach being taken is to identify job 
reading tasks that are commonly encountered on the job and in the self- 
study job training materials, and then to train those re,.'.::ag skills 
that are required to accomplish the tasks, A similar approach has been 
used to develop a functional job reading program for the Army (Sticht, 
1975b); however, the nature of the, Army's problem is rather different 
than that of the Air Force, Evidence for this derives from the fact 
that substantial Air Force technical training already involves procedures 
on such tasks as how to fill out forms and locate information in manuals: 
tasks which are taught in the Army's FLIT (Functional Literacy Training) 
program. Also, data has been collected by Dallman (personal communica- 
tion) on HumRRO 's Mechanic Job Reading Task Test. Dallman administered 
the test to thirteen airmen .prior' to their entrance into t\v^ vehicle 
maintenance course at Chanute AFB, Illinois, and found that .--iuven persons 
scored in the highest possible category and the range of score.s was quite 
narrow, llth-lAth grade 1 evels . Thus , there is the*- indication that 
attention must be focused on the higher level comprehens Ion skills. 
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It is anticipated that much of the materials developed as part of 
this effort will deal with teaching airmen to perform transformations 
on textual materials, such as transforming connected prose into' flow 
charts, classification tables, and pictures, in such a way as to more 
completely understand the materials. The working materials will be 
extracted (and- often modified) from Career Development Courses and 
Technical Orders/ Materials will be developed for use within one or two 
clusters of two or more career fields. If two clusters are eventually ^ ^ 
produced, one will represent a maintenance (aircraft maintenance and 
••repair) and the other a noa-maintenance (admini3;.;:ation/personnel) area. 
• If the prototype program is successful, it is lik^^ly that additional 
modules will be produced for at least those career fields that are 
experiencing the greatest problem. 

Specially developed job-oriented reading programs such as antici- 
pated here are one approach to AF reading problems and the typical lack 
of motivation that they inspire. It is conceivable that such a. program 
could function as a vital part of a total career education program. A 
Reading Proficiency Hnit located at Lackland AFB. might function in a 
manner similar to : :^resent n^ode- for those recruits of low basic read- 
ing skills; the job , iented program could exist as a next step and con- 
centrate on improving the conceptual behaviors of the airman wxthin a 
job context; and finally, a reading/GED-type program could provide 
access to that higher level of skill which is only attainable over rather 
lengthy periods. 

Closely related to the job-oriented reading program's emphasis on 
conceptual activities is the consideration of v/ays in which these 
acLivlties may be m^ide more efficient. The study to be described here 
has been completed relatively recently by Dansereau, Long, McDonald, 
Actkinson,. Ellis, Collins, Williams, and Evans (1975). 

The rationale for this study derives from findings that instructional 
techniques and sequences of information presentation have very little, if 
any, .impact on subsequent performance (Dansereau, Evans, Wright, Long, & 
Actkinson, 1974; Dansereau, Evans, Actkinson, & Long, 1974). Also there 
is the suggestion by Dansereau, Long, McDonald, and Actkinson (1975) that 
the effective learning strategies which facilitate a flexible approach 
toward the learning task may be a prime source of student problems. 
Difficulties in comprehension and retention were viewed* as stemming from 
two sources: "first many students tend to receive information passively 
and consequently do not actively integrate it into their existing cogni- 
tive structures- (and) second, many students apparently do not attempt 
to prodiiro multiple memory representations (that is, encodings) of the 
same ma-i.-rial in order to enhance retrieval" (Dansereau, Long, McDonald, 
Actkins(.n, Ellis, et al, 1975, p. 12). 

Following this reasoning, instructional materials tv^ere prepared for 
three techniques chat, when used by the student, w.ere expected to aid 
comprehension and retention: question and answer, paraphrase^ and visual 
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xmagery (sea Dansereau, Actkinson, Long & McDonald, 1974) . ' Each package 
also contained a section on information retrieval. Each of three groups 
Oi.- subjects was presented with a different technique package and trained 
and tested xn a series of four sessions totalling about seven and one- 
hali nours. 

Analysis of the results shows no' perf oraance differences on the 
xmaedxate post-test, which consisted of 80 true-false, multiple, fill-in- 
the-olank, and short answer items. However, the paraphrase and ima^'ery 
groups perroraed significantly better than the control group when given 
an essay type" test five days after reading the assessment passages. 
Easea on the findings and recommendations of this effor- it is anti- 
cxpated that this area of -study will be pursued using j^j-related inate- 
rxals both wxthin the conventional and job-oriented training programs. 

In summary, then, the current Air Force research program in literacy 
training involves the identification of those persons who x^ould most 
likely benefit from further training and the development of a relatively 
snort-terra program to train techniques which facilitate the comprehension 
Oi-_:too and training materials. Both efforts described witl\ respect to 
this latter categoryorely heavily on the manipulation of the forms of 
informational representation; such manipulation, theoretically, will aid 
the. xntegratxon of new mater;Lal with existing cognitive content. A 
definitive evaluation of the success of this approach ax^aits both the 
aevelopoient of the job-oriented reading program and, in the longer term, 
a valid method for evaluating ■ the operational consequences ol the 
literacy gap." 
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COMMENTS ON THE PAPER BY GROFF 

Samuel GibbOn 
Children's TeJ^vision Workshop 

Gibbon based his comments not only on Groff's paper, but also on 
the other papers/and discussion at the Conference. He expressed his . 
point of view that the Armed Services appear to be proceeding with 
enlightened programs of research and development towards closing the 
gap between reading skills of personnel and reading demands of jobs. 
However, these Services at times have had the option of screening out' 
many of the people who do not measure up to their standards. The 
choice of selection standards and the resulting implications for the 
marginal citizen may require soiQe concern by all of us about the role 
of the Armed Serv:ices in society. For instance, the view had been 
expressed in some discussifens that the criterion for determining the 
"effectiveness of and need for literacy training in the Army was whether 
or not it improved the combat readiness of the Army. Gibbon suggested 
another basis on" which literacy training in the Army might be under- 
taken: that of the Army as a place which provides opportunity for the 
marginal citizen, and that, i.s a considerable soct^T function for that 
'Armed Service to perform. Be speculated that ther^^will be a good many 
people x/no will enter the Army not knowing how to read, wel^ , ar j offered 
his conviction that people who do not know how to read waiii: very much to 
learn how! They may conceal that desire, but they indeed want i.^-. Thus,, 
because the Army has been successful in teaching many othor kills, it 
could, if it chose, 'teach these people who desperately want to learn to 
read but have not, how to read. Gibbon suggested that there are many 
ways in which reading training could be done, but pointed to the need 
for a values orientation such that developing the ability to read a 
simple sentence might have the same importance in basic training that 
a shoeshine does! 

Pointing to the success of television and other electronic media 
in providing private ways of learning for people whose shame at their 
ignorance is extreme, Gibbon suggested that use of such media in literacy 
training might be explored in all the Services. Finally, returning to 
Groff's paper. Gibbon commented on the observations in that paper which 
indicated that the operational consequences of a literacy '"gap" have 
not been identified, and expressed his belief that discovering those 
consequences might contribute to the program of ongoing research and 
development. 
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ONGOING RESEARCH AMD DEVELOPMENT 
IN READABILITY AND USEABILITY OP TECHNICAL WRITING 
IN THE ARMED SERVICES 
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INTRODUCTION 



The preceeding section dealt with one jnain approach to literacy 
problems in the Armed Services': R&B to produce better readers through 
literacy training. This section deals with the second main approach: 
R&D to produce better reading materials. 

The section begins with Tom Jr'^rran^s review of readability research 
in the Navy» He begins with a discussion of the terms "readability" and 
"coinprehensibility" and of the interaction of readability, comprehensi- 
bility, and useabilit: with each other and with various- characteristics 
of the readers* 

Currari discusses both the prediction of readability and the product- 
ion of readable writing. The ^irediction of readability is attempted 
through application of a readability formula. Curran describes the nat- 
ure and application of five readability formulas and considers' the special 
problem involved in assessing the readability of technical material and 
the tradeoff between the difficulty of computation of multivariable 
foimiulas over that of one- or two-variable formulas and the resulting 
increase in predictive pox^er. In addition, he discusses non-statistical 
methods of measuring difficulty, especially the cloze test measure of 
comprehensibility. In describing the process of producing readable 
witing (including the role of prediction), Curran discusses the role of 
style guide and military specif ications • He describes several automatic 
and computerized systems v/hich can be used to aid the writer by automating 
the process c ' prediction. 

The focuij shifts to comprehensibility as Art Siegel reviews Air Force 
research and development in that area. Such R&D injcludes attempts to use 
auditory supplementation to increase comprehensibility and the development 
of new comprehensibility measuremcrat, concepts which focus on the cognitive 
and language processing load which a text places on the reader. Siegel 
conclud'^.iG with a description of a comprehensibility measurement computer 
prograuii. 

Dick Kern reports on the Army's R&D efforts not only on the reading 
difficulty of Army literature, but also on its content and orientation and 
its useability. Kern describes studies which investigated the relation- 
ships between the reading ability of nrjrsonnel and the reading difficulty 
of the manuals they were expected to use, their use of manuals on the job, 
and the quality of their performance on job saiiiple tests. He reports on 
research which indicates th^t Army training literature in the past has- not 
always been oriented toward its users, both in content and in orientation. 
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The Army is attempting to develop more user--oriented literature, . as 
indicated by a user-oriented approach in a guidebook for. Army writers and 
by the user-oriented publications of CATB% Related to the question of user 
orientation of a manual is the question of its useability. As Kern 
reports, luost p?st evaluation of manual effectiveness has been in the 
content of entire training programs* Kern suggests that future research 
on the useability of printed material focus on the identification of char- 
acteristics of printed material which influence useab51ity followed by 
the development of methods to assess useability of materials. 

Much of the Air Force R&D on the useability of job reading materials 
has been connected with the development and evaluation of job performance 
aids x-zhich are various printed materials designed to enhance on-the-job 
performance, especially of maintenance and troubleshooting tasks. In 
reporting on those efforts^ Robert Johnson discusses specifications for 
the development of useful job performance aids (i.e. the steps to take 
to ensure the development of xaseable data) and the resulting character- 
istics of the completed JPA s which contribute to their useability. 

Job performance aids have also/.been a part of the Navy's R&D on the 
useability of technical manuals. Bill Muller, in his rev'iev; of current 
Mavy research on useability, graphically shows the increasing magnitude 
of the production of technical u.anuals» He discusses the key elements in 
the Navy's technical documentation system and some of the problems of the 
system with regard lo the development of useable technical documentation- 
Nuller concludes his report with a review of four ongoing offer ts which 
attempt to provide assistance for the technical writer. 
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readabilit:^ research in the navy 



Thomas E. Curran 
Navy Personnel Research and Development Center 

A great deal of conceTm has recently be'ea voiced witH regard to the 
quality of instructional and maintenance manuals in the Navy. Men, both 
on the job and in the classroom, have tended to have problems using 
manuals due to the difficulty of both text and graphics, inadequacies ' 
m content, lag time in updating, etc. The thrust of this paper is with 
the first of these deficiencies, but its concentration differs from that 
often found in work on readability . Readability, per se, is not a diffi- 
cult problem if one assumes as its operational definition a "readability 
formula score." As measuri^d by readability formulas, almost anyone can • 
write readably. One simply uses short words and short sentences. Given 
these, the "readability" of the material will be at a high ("easy") . level 
when indexed by the most common formulas. Obviously, this is a simplistic 
view, particularly for technical, writing. Technical writers (who encom^ 
pass nearly all writers preparing Navy materials) cannot always obey the 
dictum to use short words, What "easy" word can be substituted for 

oscilloscope," for example? But: even for these writers, readability need 
not be a severe problem. One must merely modify the above prescription 
to say "use fcaniliar words and short sentences" and modify the readability 
assessment procedure so that long familiar words do not inflate the reading 
grade level (RGL) of the material. A major part of the R&D effort in the 
field of readability should therefore be directed at identifying long words 
which are known to be familiar to the intended audience and effectively 
transmitting this information to the writer. This topic will be discussed 
in other contexts at a later point. 

There is a second characteristic of writing that deserves special 
attention- comprehensibility. There seems to be some, misapprehension 
in the literature to the effect that high readability ensures comprehen- 
sibility. Only if readability is taken to include good "style," smooth 
flow of ideas, avoidance of complex sentence structures, etc. is this 
the case. Note that this represents a rather drastic departure from the 
operational definition of readability suggested above, and, in fact, 
closely resembles the accepted definition of comprehensibility , But the 
position is taken here that they are clearly two diff erent . concepts . At 
this point in time, readability formulas cannot adequately assess charac^ 
teristics such as style and sentence structure. It is true that read- 
ability and comprehensibility often go hand in hand. The nursery rhyme 
"Mary Had A Little Lamb" is written at a very low level of difficulty, 
and because of the structure of its sentences, its simple words and its 
easy flowing style, it is also quite comprehensible. But examine the 
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other extreme, (unlikely as ic would be comirg from any rational author). 
If the words in the rhyme wera ^'scrambled/' with the length of sentences 
left intact, its readability score according to fon~ula would be identical 
to- that of the original version. Yet its comprehensibility would be 
reduced enormously. In between these two extremes fall countless examples 
of writing which vary along each of the two dimensic /s. In short, compre- 
hensibility, like readability, is a property of the written material, and 
when the work is done carefully the tx^o are at least moderately correlated. 
But it would seem, prudent to continuously bear in mind that the former 
cannot be directly assessed by means of existing formulas. 

To further complicate the issue, there is really no point: in speak- 
ing of either readability or co:::prehensibility unless the iulended 
audience is taken into account. Two characteristics of the audience 
(other than general reading ability) are of prime importance: the back- 
ground knowledge possefj.sed^ and the degree of motivation involved. The 
adult reader faced with the scrambled version of "Mary Had A Litcie Lamb^' 
might comprehend it reasonably well because of repeated exposure to it 
in the past, and might find it to be a stimulating (i.e.> motivating) 
puzzle-solving experience. The sane strings of words seen by a child for 
the first time would probably be incoiaprehensible, even triough the 
individual words themselves could be decoded. 

Lest one be mislead by the above statements, it sliould be made clear 
that readability is NOT a trivial issue. It is a mcensary , but not a 
sufficient condition for high-quality . manuals . All other factors being 
equal, COI-fPREHENSIBILiri: requires high READ.^BILITY , and one step removed, 
USEAJP.ILITY requires high COMPREHENSIBILITY. If a manual is not readable, 
the student or technician will not even attempt to use it once he has 
been stymied by its difficulty at the outset. He probably, will avoid 
it from that time on, just as would anyone presented with a tome replete 
with long words which he cannot decode.. Bu*- to be readable is not enough. 
If the manual is not also comprehensible, the man possibly will not be 
able to use it despite his need for the information it contains, /aid the 
problem again is not even that simple. Material can be virtually unread- 
able according to a formula score and yet one who is vitally interested 
in the topic and/or needs the information regardless of the effort 
required to obtain it may slave his way through it despite the writer^s 
opposition. 

Tlie interactions among, readability, comprehensibility, and useability 
(all properties of the matevial) and reading ability, comprehension 
ability, motivation, and background experience/knowledge (all properties 
of the user) should provide the overall model for any R6tD effort in 
improving Navy manuals, 

In the third quarter of Fi.^-cal Year 1975, the Navy Personnel Research 
and Development Center (NPRDC) was tasked to- investigate the area of the 
readability and comprehen.sibility of technical manuals (TMs^< . An 
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extensive suirvey of the literature was undertaken to determine the state- 
of-the-art in this area and report findings (Curran, 1976). The con- 
clusions of this author were that the area of readability and compre- 
hensibility has been covered in breadth but not in depth. A dilemma 
appears to have evolved. Formulas can be applied efficiently —automation . 
has become the rule— but it is possible that only judgments of the mate- 
rial by the readers can provide an index of its comprehensibility . 

PREDICTION OF READABILITY 

This author makes the distinction between prediction of readability 
and production of readable writing as put forth by Klare' (1975). This 
section deals with the first of these problems while the following 
section deals with the latter and the interaction between the two. 

Readability Formulas: General 

The most common method for predicting the difficulty of a given 
piece of material is the readability "formula." A great many of these 
have been developed,, with the majority using as variables some index of 
sentence difficulty and individual word difficulty. Reviews of these 
formulas up to 1960 can be found in Klare (1963); Klare (1974-1975) 
reviews those developed after 1960 and modifications to earlier ones. 
Preference in this paper will be made to only five such formulas: the 
FORCAST formula, the RIDE scale, the Fog Count, the Flesch Reading 
Ease (RE) formula, and the Dale-Chall formula. 

In general, the prediction of readability by formula involves 
"counts" of various components of written material and, using passages 
of knoxm difficulty as criteria, computing a- regression equation on the 
counted characteristics. Commonly, the criterion passages are develops rl 
using the "cloze" technique. This procedure, which is of concern thr-^ 'i^ " 
out this paper, involves extracting verbatim passages from the mater i^^^ 
and deleting every nth word, repl-acing each with blanks of standard 
length. Normally, every 5th word is deleted, resulting in five v.-^r- Lons 
of the test for each passage (deletion of words 1, 6, 11, ... n, u i*o 
10, 15 ... n) . Thus, each word in the passage is deleted once acrcr-s z\i:-: 
five versions. . Subjects of known reading ability rare asked to fil"' An 
as many of the deleted words as they can, without having first rea/. ^.n*- 
intact passage. The reading level at which 50% of subjects can f ii I ii' 
approximately 40% of the missing words is usually taken a'; the read i., 
grade level (RGL) of that passage. ^ This "scaled" RGL is then used 
the. criterion for regression of the counted variables. 



'RGL is comparable to, but not equivalent to. school grade; .t is a su'V.e- 
what arbitrary level at which a pan malar grade student shoutd be able 
to read with satisfactory comprehension. 40% correct cn the cloze test 
has been shown ' be approximately equi.valent to a 75% s. )re on a 
multiple-choic. . test on the material. 
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As an example, Borrauth (1969) used this procedure for deriving a 
number of readability formulas. He examined, in an extensive correla- 
tional study, approximately 170 different variables— such as vocabul.i.ry ^ 
syntactic complexity, and parts of speech— and their relationship to rh«: 
difficulty of ^^yritten material. He first determined the correlation 
each of these variables with the difficulty of 330 lOO-x^ord passages as 
scaled with the cloze technique. He then entered the variables Iulo a 
regression equation to determine which of them in combination hesi. ,rje- 
dicted the difficulty of the passages. The outcome of this study 'Js a 
series of formulas, each designed for a different purpose, incorp... .ating 
the "best" of the variables for predicting the difficulty of otaer writ- 
ing . 

The Flesch "Reading Ease" Formula 

Probably the most wJ.dely used (and most consistently powe^:!..;) of 
the readability formulas is that developed by Rudolf Flesch (1948'- 
This formula - termed the E.eading Ease (RE) formula- uses as vai i.ables 
the number of words per sentence and the number of syllables . per LOO 
x>7ords of text- The original RE formula is presented In Table l.:. . Tn.is 
formula vzas developed in much the same manner as described .^.bc/e, except 
that the criterion was a set of standardized passages raLher v-.iian passages 
normed for the specific purpose by the cloze technique. The forTT^iiia 
indexes the difficulty of material on a- scale from 0 (practic iLlv -...-ireac?- 
able) to 100 (extremely easy). Using this index, the RGI^ c.n ; d-.^"-! • 
mined from a conversion table. The RE formula was developed and v^*l j d^i/'^v-d 
on children and civilian adults , with reading material appropria;:e ro 
thosi'^ samples, and its usefuine.s^ for assessing the difficulty oi: r)i;.lcary 
t^'chn-i-Ccvl or in,r- ^ructivmal writing is therefore suspect. In ordL.r lo 
o»Jercor.;ie thi-^ problem, Kincaid. Fishburne, Rogers, and Chiss'. /a (J ^175) 
recalculated the formula using Navy enlisted personnel readiug y job- 
relevant literature,^ In the process of this recalculate c .i^ RE formula 
was also revirjed to produce RGL directly without having L^. :£.:.e the extra 
step ol converting ''reading ease" to RGL via a conversion *'-.ble. The 
recalculated version of the RE formula is shown in Table :.b . The RE 
forr.iula has the advantage of being r j. Latively simple to tS'-uaipute manually 
and is adaptable to automation as well. Kla^e, Rowe, St- John, and 
Stolurow (1969), among others, have developed a computer program which 
provides r.he RE index baf ed on the original i-'lcsch formula. It remains 
a fai.rly si.mple ta^^k to revise, the program for the i ecHlculatcd version 
of the formula. 



These authors also recalcu'Jp'ed two other formulas- the •■'i.tomated Read- 
ability Index and the Fog Coint— which will be c^r^dressed at a later 
point. 
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TABLE la 

ORIGINAL FLESCH READING EASE FORMULA 



READING EASE = 206.835 
- 1.015 
.846 



(words/-. o.-ttcn^'^) 
(syllables/lOO words) 



TABLE lb ■ 
RECALCULATED FLESCH READING EASE FORMULA 



GRADE LEVEL = .39 (words/sentence) 
+ 11.80 (syllables/word) 
- 15.59 
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RECRUIT READING ABILITY (N=21,000) vs RTM DifFICULTY (N=1851 
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Figure 1. Comparison of Recruit Reading Abilities 
and' Training Manual Difficulty. 
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RECRUIT READING ABILITY (N=2r000) vs RTM DIFFICULTY (N=185) 
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Figure 2. Comparison of Recruit Reading Abilities 
and Training Manual Difficulty 
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Application of the Reading Ease Formula 

Based on its consistent validity when compared with other formulas 
and its relative ease of manual computation, the Chief of Naval Education 
and Training Support "Readability Working Group'' conducted an analysis of 
all Navy rate training manuals (RTMs) to determine their difficulty 
using the RE formula. Biersner (1975) reports on this effort. He analyzed 
195 Navy RTMs, using both the original Flesch formula and .the revision by 
Kincaid, et al, (1975), (As with Biersner, this formula will be referred 
to herein as the Kincaid formula.) Biersner 's work illustrates vividly 
the importance of the ptediction of readability in general and- to the 
Navy in particular. . Bear in mind, throughout the discussion to follow, 
that RTMs are a major source of information required by Navy personnel 
for advancement in rate. Twenty-five samples were taken from each of 
these manuals, their difficulty was determined, and the average for the 
entire manual was computed. Considering only the Kincaid formula (which 
is considered more appropriate for Navy writing) , the range of difficulty 
of the 185 RTMs was from an RGL of 8.82 (for Basic Machines) to one of 
16.26 (for Disbursing Clerk 1 & C) with a median RGL of about 12.6. That 
is to sayj 50% of the RTMs were written at a level of difficulty (as 
indexed by the formula) such that a "beginning college reading level" 
would be required to understand them. This should be interpreted with 
caution, however. There xs no one-to-one correspondence between the 
difficulty level indicated by a readability formula and the ability of 
a person to profit from the written material. Biersner makes this point 
when he says, " — the relationship between RGLs (as determined by any of 
the available readability formulas) and reading comprehension or per- 
formance effectiveness is not well established, despite the importance 
of reading to the development of most other skills" O^iersner, 1975, p. 7). 
It may well be that the mere presence of a Navy recruit in the naval 
milieu may be sufficient for him to comprehend the necessary elements of 
the RTM required for advancement. But this does point up the issues 
raised in the introduction. A formula can index the readability of mate- 
rial, but (at least as yet) no formula can index its eomprehensibility . 
It is clear that the value of writing can be accurately determined only 
from within the framework of an overall moder- the readability and the 
state of the user in terms of motivation, background knowledge and 
experience, etc. But at face value, assuming all other factors to be 
equal, the comparison between the RGL of the manuals as reported by 
Biersner and the reading abilities of recent Navy recruits (see Duffy & 
Nugent, 1975) shows a quite apparent "man-manual mismatch." This com- 
parison is presented in Figures 1 and 2. Figure 1 indicates the full 
range of the RGLs of the RTMs from the one manual at the 8th-grade RGL 
to those at the 14th and above. Reading abilities of recruits (as 
measured by the Gates-MacGinitie test) are shown for the same intervals. 
The fact that no reading abilities are shown for levels above 12th grade 
is an artifact, due simply to tie fact that the ceiling for this test is 
at the 12.0 level. Figure 2 presents the same data in a different 
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perspective. Here, reading abilities are indicated by percentages from 
grade 4.Q iand below) to grade 11.0 (and above). All RTMs whose diff:.- 
culty exceeded 11.0 according to formula score are bulked into the 
latter interval. Noting the cumulative percentages (again at face value), 
it would be" possible that approximately 24% of these recruits could read 
only one RTM — the single manual in the sample below the 9.0 level. To 
interpret further, approximately 82% of the RTMs are written at the 11.0 
level or above; only 45% of the recruits were tested as reading at the 
11.0 level or above. And one further illustration of the mismatch (not 
indicated in the figures) deals with the manual Ba^io Military Require- 
merits. The RGL of this manual is 10.85- Knowledge of its contents are 
required for advancement to £-3. Yet its difficulty level is approxi- 
mately the same as the median reading level of the over 21,000 recruits: 
It is possible that 50% of the entire sample would not be able to read 
this manual. Clearly, our ability tc predict the difficulty level of 
written materials is of immense value. The data discussed above gives 
us every indication that certain RTMs should probably be revised downward 
in difficulty level unless reading abilities can be increased. 

In addition to the striking findings reported above, another feature 
of Biersner's work deserves mention. As noted earlier, the computation 
of the reading ease formula involves a count of syllables, words, and 
sentences. While this can be accomplished relatively simply with nothing 
more than a paper and pencil, Biersner reports on a device which greatly 
facilitated the giant task of his analysis. Biersner directed the devel- 
opment of an electro-mechanical counting device which would permit more 
rapid and reliable gathering of the data. This device took the form of 
a "stylus" which, when pressed, to a surface, trips a microswitch, xvhich 
in turn activates a counter- A relatively unskilled person using this 
device can therefore press the stylus to the working copy at the end of 
a syllable, word, or sentence, and that variable would be indexed by the 
counter. Biersner reports that " it made data collection over 30 per- 
cent faster, while maintaining high reliabilities" (Biersner, 1975', p- 
17). 

Technical Terms and Measurement of Difficulty 

A major problem with most readability formulas (as alluded to above) 
is that they were developed and validated using either school children 
or civilian adults. There is good reason to believe that the abilities 
of a child reading at the 6th-grade level (for example) and of a Navy man 
also reading at the 6th-Tgrade level are quite different- Again, the 
military environment itself may be expected to account for perhaps great 
differences in the words with which the person is familiar. This problem 
is even more acute when one is concerned with technical T^^riting. Since 
word difficulty is usually indexed in terms of the lengnh of the word, 
the technical terms encountered on the job will tend to inflate read- 
ability formula scores. Yet is is probably nonsense to assume that an 
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electronic technician would be unfamiliar with the word "oscilloscope/^ 
Traditional formulas have nof addressed this problem directly, and y--*r 
they must. This is central to the ongoing theme of this paper. The 
fact that a readability formula indexes, a piece of writing as "easy" 
or "diff icailt" does not ensure that it will' appear that way to the 
intended user. Two questions seem, to be at issue here: to what degree 
do technical terms inflate the analysis (since they seldom exceed about 
15% of the total number of words), and, if indeed they are a problem, 
by what means can the difficulty of technical writing be roalistically 
indexed? The first of these questions is a basic empirical one, and 
should be answered before proceeding with R&D involving the second 
question. Additional formulas, and measurement considerations not 
involving formulas, will be> discussed below, taking these R&D priorities - 
into consideration, ' , . 

The FORCAST and RIDE Formulas 

Caylor, Sticht, Fox, and Ford (1973) made an attempt to allay the 
problems of using formulas derived from civilian samples in determining 
the difficulty of military writing. They examined a' total of 15 struc- 
tural variables in Army training literature, including words per sentence, 
number of independent clauses, number of one-syllable words, and total 
number of syllables. They were also dissatisfied with the traditional 
readability formulas fur use with military writing: 

The fact that formulas have validity coefficients of about 
,70 for predicting the performance of school children on reading 
comprehension tests indicates that they account for roughly 50% 
of the variability in reading performance of ahildi'en. It is 
likely that they may account for less variability in adult per- 
formance, especially since material containing large numbers of 
technical terms would increase the estimate of difficulty made 
by the readability formulas, (Caylor-^ et al, , 1973, p, 6,) 

They therefore examined job-relevant mat'::r\als from seven militar^v 
occupational specialties (MOSs) in the Army, using scaled RGLs of this 
material (based on the cloze test) as a criterion, and computed regression 
equations with the 15 variables. They found thac the number of one-syllab 
words alone correlated ,86 with the criterion cloze scores, and that the 
addition of either one or two additional variables to the equation 
produced no significant increase in this correlation. They therefore 
included only this variable - number of one-syllable words - in their 
formula, which was termed "FORCAST," This formula became: 

FORCAST RGL = 20 - number of one-syllable words 
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Based on their results, ^and considering that this variable is probably 
the most simple count to achieve reliably, the rationale for recommending 
the one-variable formula seems sound. It should be noted, however, that 
others have found that the addition of a sentence-length variabl|^ adds 
considerably to predictive power. Kincaid, et al. (1975), for example, 
found that the addition of a sentence difficulty factor to the word 
difficulty variable increased the "coefficient of determination" (indicat- 
ing the degree of shared variance between the predicted grade level and 
comprehension of the test passage) from 41.6% to 57.2%. This is a sizable 
increase in power, indicating that the findings of Caylor, et al. might 
be reexamined to determine if the difference in power outweighs the ease 
of .computation. 

Carver (1973^:5 1974) also developed a single-variable ..formula, which 
he termed the RIDE . scale (an acronym for Reading Input D^ifficulty Estimate 
'^^sriable used in this formula is simply the average number 
letters per work (Ipw) in a passage. There are five levels which index 
RiDP: difficulty: Level 1 (up to 4.0 Ipw), Level 2 (4.1 to 4.5 Ipw) , Level 
3 (4.6 to 5.0 Ip^O, Level 4 (5,1 to 5.5 Ipw), and Level 5 (5,6 Ipw and 
above). This formula would appear to suffer from the same deficiency as 
tho FORC AST formula V TLat as, it considers only word difficuic/ and 
ignores sentence length. Further, it remains to be seen how this formula 
v/ouid hold up with technical vriting containing long, but familiar terms, 
in view of the fact that it was validated on the Miller-Coleman passages 
(See Aquino, 1969) using school children. Tnis question should be tested 
to determine its validity. 

Carver based his work on che findings of Bormuth (1966, X969), which 
showed that the number of letters per word was one of the highest single 
correlates of the cloze difficulty of 330 100-word test passages. Accord- 
ing to Bormuth (1969) only the Dale "Long List' correlated more highly as 
a single variable than number of letters per word (and that by only .006). 
It should be noted, however, that these variables both- deal only with 
word length/familiarity, and do not touch upon sentence difficulty. It / 
has been shown consistently that word length or familiarity accounts for 
a greater amount of variance than dees sentence length. If one's major 
goal. is to devise a prediction formula zontaining only ^ne variable, 
therefore, the rationale for the RIDL scale would be sound. It seems 
worthwhile, however, to pursue further the potential increase in predictive 
power with the addition of a sentence difficulty factor 

Bormuth (1969) did go beyond the examination of a single variables 
in isolation. He conducted an extensive ^correlational study using 
approximately 170 different variables, of ^ which Ipw was one. As a first 
step, he determined the correlation of each of these variables single with 
the scaled .difficulty .level of passages as referred to above. However, he 
then went on to determine, using multiple regression techniques, which 
of these variables in combination yielded the best predictive power. He 
arrived at four basic passage length formulas (each of which was computed 
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against four different criteria) ► Tw.* of these formulas. .(termed "unre- 
stricted") employ large numbers, of varia'':.L23 and" are clearly unsuitable 
for other than very sophisticated -.ae.O!La'»-ad computation. (Interestingly, 
neither of these basic formulas includes Ipw as a variable.) To 
exemplify these formulas, and consider* ing the fact that automation 
seems obvious to become the rule for the ultimate analysis of difficulty, 
the "short form" of the unrestricted formula will be described briefly. 
This formula consists of eight different variables plus the square of 
two of these, totaling 10 entries in the fomnila. The eight basic vari- 
ables are (1) number of Dale Long List words, (2) letters per minimal 
punctuation unit (and its square), (3) number of "referential repetition 
anaphora" (and its square), (4) number of numerical nouns, (5) number of 
derived adjectives, (6) number of common nouns, (7) number of relative 
clauses, and (8) number of "class inclusive anaphora." It seems obvious 
that the present state-of-the-art of readability analysis does not per- 
mit efficient application of such a complex formula. Its co'rrelation 
vith the mean cloze criterion (.874), however, is exceeded only by that 
of the even more' complex basic unrestricted formula (.889). The question 
remains (even disregarding computer state-of-the-art) whether these 
correlations represent diminishing returns over the somewhat lower 
correlations of the much more simple manual and macfnine computation 
formulas described below. 

Looking at the letters per word variable alone (as used by Carver) 
and comparing it to che two Bormuth formulas which include it and other 
variables, the results are as follows: 

(1) Letters per word alone correlated -.721 with mean cloze scores. 
(For the purpose of this comparison only mea?i cloze score will be examined 

(2) Letters per word, when combined with letters per minimal 
punctuation unit, in the manuat computation formula, correlated .808 with 
mean cloze score (and .833 in cross-validation-^). 

(3) Letters per word, when combined with number of Dale Long List 
words and words per sentence in Bormuth 's machine computation formula, 
correlated .833 with tne criterion (and .920 in cross-validation). 

Disregarding the fact that each of the correlations increased in 
cross-validation, the addition of new variables to the single Ipw vari- 
able resulted in an increase in observed (or shared) variance of from 14 
to 17%. It remains to be seen whether the additional difficulty and 
possible added unreliability in counting letters per minimal punctuation 
unit or words on the Dale Long List would of f'set this rather sizable 



Cross-validation was accomplished by applying the formulas to 20 passages 
of 275-300 words each taken from earlier work by Bormuth (1969). 
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Increase in predictive power. Carver reporrs correlations of ,93 and 
-•94 between the RIDE scale and Dale-Chall and Flesch formilas, respec- 
tively, using the Miller-Coleman passages as criteria. He concludes that, 
fov these vassages^ "... there" seeas to be little difference betxvreen the 
predictive" ^ralidity of Dale-Chall, Flesch, and RIDE" (Carver, 1974, p. 
12)^. These discrepant findings — both of the other formulas consider 
sentence length md word difficulty or familiarity — remain to be explained- 

The Fog Count 

The Fog Count, developed originally in 1953 by McElrcy for the Air 
Force and described in 2. Guide for Aiv Force Writing (AFM 11-3), is per- 
haps the most simple two- variable formula to compute manually. It involves 
simple counting the number of words of three or more syllables, the total 
ntimber of words, and .the number of senteaces and entering them into a 
simple formula. Rather than discuss the original Fog Count, it would seem 
more" appropriate to addresr, the "recalcul'^ted" Fog Count presented by 
Kincaid, et al,(l975). According to these authors, the revised version is' 
"very similar" to the original except that a different subtracted constant 
is used to redress a problem of over-estimation encountered with the 
original. The Kincaid, et cl. recalculation was accomplished using 569 
Navy enlisted personnel reading material from Na\'7 rate training manuals, 
at least some of which was technical in nature. In both the original and 
the. revision, long familiar tenrtS arc treated specially to avoid the prob- 
lem of their contributir.g errnneously to the difficulty of the material. 
For example, "General Eisenhower" (consisting of a total of seven 
syllables) is considered as one "easy" word for the purpose of calculating 
the Fog Count. Other units, such as numbers and some abbreviations are 
treated in a similar manner. It would seem that a similar procedure could 
be applied in dealing with technical terminology known to be familiar to 
the user of a given piece of material. Here a problem enters that is 
related to the first of the basic questions posed above. Before the Fog 
Count (or any other formula) can be made to take account of such terms, the 
terms themselves must first he identified. Hiis is not so easy as one 
might first suppose. In the field of electronics, for example, a basic 
dictionary of terminology for comprehension of written material could be 
obtained from an examination of books in the area. But to whom will this 
dictionary apply? Will it be sufficient to have only one such dictionary? 
If there is only one, will it apply equally as well to the difficulty of 
material designed for basic electron^L^ craining (e.g., "A" school) and the 
senior petty officer on the job? Considering these questions, the research 
involved would seem to be a difficult, but not insurmountable effort. 



Carver points out, however, on the basis of later work, that the Flesch 
and Dale-Chall formulas seem to b^ "consistently" better than others. 
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And once the dictionary (ies) is (are) constructed, there is the question 
of whether the additional effort required in applying them, is accompanied 
by a significant and practical gain in our ability to gear materials to 
the desired level of difficulty. With regard to the Fog Count, and pre- 
suming that a dictionary of technical terms proves worthwhile, the present 
advantage of easy manual calculation will probably be reduced- The effec- 
tive use of a dictionary of technical terms will virtually demand the 
automation of the predictive device- .The 'Fog Count, in manual form, 
however, might even then be of value, for a ''rough" difficult::/ analysis 
where automation is not available. Reference to a "short list" of, say 
the 25 most common terms, could be accomplished during a manual count, 
and such words treated as "easy" for the purpose of computation* Further 
reference to this problem will be made with regard to the Dale-Chall 
foriuula below and' in the discussion of "production" of readable writing. 

The Dale-ChaH Formula 

Tl\e Dale-Chall formula (D£«le & Chall, 1948) is a two-variable formula 
which is based on average sentence length and the number of words in the 
passage not on the "Dale List of 3,000 Easy Words." (^fhis "Dale List" was 
compiled by asking fourth graders which of a number- of words they "knew.") 
To the best of this author^s knowledge, the Dale-Chall ' (D~C) formula has 
not been validated in. a military setting. It is discussed here bo'cause it 
has consistently been found to be highly valid v;hen compared with other 
formulas in non-military settings, and because it offers a ''model" for 
construction of a technical dictionary or lookup list such as discussed 
above. Again, presuming that the appropriate technical terms have been 
identified, such terms could possibly be simply added to the Dale list^ 
Then, when encountered in text (since they are on "the list"), they would 
be considered "known" or "easy" and not contribute to the difficulty index 
of the passage. If research indicates that a technical dictionary does 
improve our prediction ability for military writing, the D-C would be 
recommended for use due to its "track record" of validity. 

Other Methods of Measuring Difficulty 

The above discussion has considered only the prediction of readability 
by basically "statistical" methods. Mention should also be made of two 
other methods for determining the readability and comprehensibility of 
written material. The first of ^these involved a judgment of the quality 
of material by persons who read a passage and compare it to some standard 
or to their omi experience of job reading requirements. Carver (1974) 
examined this procedure usi.ig his "Rauding Scale." A major advantage of 
this method over that of formulas is that style, sentence- structure, etc. - 
can be assessed by the reader, a task which formulas are unable to carry 
out. In Carver's approach, judges who qualify on the "Rauding Scale 
Qualification Test" are asked to judge sample passages and • categorize them 
according to a set of six "anchor passages." Using the aver>-?ge rac ing of 
three su-". judges, a grade level for the target passage is determined. In 
short, C-.rver found the correlations between Rauding Scale judgments and 
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the RIDE, Flasch, and D-C formulas to range from .74 to ,84. These 
moderately high correlations indicate than human judges axp. taking into 
account characteristics of writing similar to those used in calculating 
the formula scores. It seems equally obvious, however , that human judges 
are considering "something else" in their ratings; what this might be 
one can only surmise. Carver (1974) hypothesizes that the Rauding Scale 

reflects " the difficulty of the ideas or concepts in a passage," and 

indicates when ".,. choppy sentences and inappropriately inserted little 
words" make material harder ro read and understand (than a statistical 
estimate would indicate). The major disadvantage of the use of h^:man 
judges in the assessment of readability and comprehensibility is simply 
availability—availability of time and of personnel. For such a pruce-- 
dure to be used effectively with the vase amount of material with which the 
Navy is concerned would seem to be prohibitive in the extreme,. 

Another method for determining the" quality of written material is the 
"cloze test" referred to above in the context of readability criteria. 
There seems to be considerable disagreement as to what the cloza test 
actually measures. Some say it measures readability; others use the terms 
comprehensibility, comprehension, or understanding. Kincaid, et al. (1975) ^ 
for example, state that "...subjects were tested for their ur.devGtanding 
of the selected passages using the close procedure" (p. 3, italic? added). 
Sticht (1975c) says "...the cloiie test provides a valid measure of reading 
comprehension'' (p. 20, italics added). A number of writers (inclading 
Taylor [i953], who developed the procedure) refer to the cloze technique 
as a measure of readability. 

If the operational definition of readability is taken to be the index 
provided by a readability formula, the cloze test will not consistently 
measure readability. The results of a cloze test on the "scrambled" 
version of "Mary Had A Little Lamb" (for example) would probably bear 
little resemblance to the low difficulty predicted by a formula. Clearly, 
this is an extreme case. I-Jhen writing of fairlv high quality is examined 
there is consistently a moderate to high correlation betw.en cloze per- 
formance and readability as predicted by formula. But one would expect, 
as the. general "quality" of writing decreases, this correlation v/ould 
correspondingly drop. It does not seem wise to use as a measuring device 
such a "rubber yardstick." 

A correlation is also consistently found between cloze scores and 
"comprehension" or "understanding." It is fairly well established that 
.a cloze score of about 40 45% corresponds to a comprehension test score 
(after reading the material) of about 75%. But again, :'.t can be shovm 
that there are limiting factors. This author has condyle ted a relatively 
modest pilot study in which cloze tests were administered on a single 
passage of about 200 words. Subjects were then asker to state as pre- 
cisely as possible what the passage was all about. inly 3. jubjects ouf 
of the approximately 50 tested was able to provide /iven "rca.-onable" 
approximation of the passage content. Yet the mear cloze sco>-e over all 
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subjects was approxinately 40%. This would seem to be al. least an indica- 
tion that the subjects ware no-t ^'comprehending" what they were reading. 
It is anticipated that a follow-up study will examine the differences in 
performance when (1) a title descriptive of the content of the passage 
is provided and (2) a traditional t.est of comprehension is administered 
following the cloze test. Again, since there are clearly limits beyond 
which the . cloze-comprehension correlation will not hold up, should we not 
be more precise in speaking about cloze in general'^ 

It is held here that close measures conrpv^^hensihi-litij . Klare, 
Sinaiko, and Stolurow (1972) take a siuil'j.r position when they ' :y, 
cloze measures the relative comprehensibility of intelligibility of 
written material." To reitera'ze an earlier statement,, readability and 
comprehensibility are not one and the same. Compreheasibll Lty entails 
such factors as writing style, sentence structure, and expectancy for 
familiar words. The first two of these factors cannot be detected at all 
by available readability formulas. The latter can be tc socie extent, 
although the identification of those words v;hich 'ira fac:*.'.'. '"a^ to a given 
audience has not yet been satisfactorily accomplished. Y>. .•::iese factors 
(and undoubtedly others, as well) are critical to perf amiau..:^^ a cloze 
test. Multiple '"embedding/' for e:-:ar-iple, would nost likely cloze 
performance, to deteriorate greatly. Consider the following . -^Lice : 

"This is the malt the lat the cat the dog teased kjJlei ^.e 
that lay in the house that Jack built." 

In this exampl .^— a syntactically correct sentence with a vo La-: J.vely 
■ low level of diffi^ t:. ty accord li'ig to formula — a person encoun'T :ir ine thr: 
sentence for the first time would likely have a very diffimlf time with 
a cloze version. Style — that undefinable characteristic C)f wr i tlug — will 
cause even greater problems. There seems to be little, doubt that the 
smooth flow of ideas, with few departures from the "standard" syntax, 
enhances the probability that the x^/ord which occurs to a reader for a 
given "set" will be the correct one. In short, it is considered that the 
cloze test should be considered a supplemenv to a readability formula. 
Given that a passage is readable (as predicted by a formulc-.), the close 
performance of a set of subjects similar to the in-:2nded audience of the 
passage should ensure that the material is also o.^pr. heuslble . 

Assessing Job Reading Requirements 

Formulas, judgments, and cloze sc< .ro all clearly play a role in 
determining the level of ability requi ..:d by a user of written materials. 
They do not, however, give any indicatl." i^f the x.^ading ability required 
for a specific job. or even of the reading required for that job. The 
Navy Personnel Research and Development Center (NPRDC) and the Human 
Resources Research Organization (HumRRO) recently set out to eek answers 
to thecse important questions. This research effort involves the dir', ct 
assessment of job incumbents* ability to read and uad . .rs tai)d job reading 
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naterials. Ic involves a sLractu^ed intcrviev; on che jpb site in which 
the men are asked to report what specific reading they have undertaken 
in the previous day or so that was directly connected with their work. 
. I-Taen a loan reports that he ed :i st;acific portion of a specilic publica- 
tion to perfom: a specific job, he is asked to retrieve ::hat same infor- 
mation in the presence o"" interviewer. From this interview '-.lata, a Navy 
Reading Task Test (NRTT) battery will be constructed, consisting of mate- 
rials which job incumbents reported that they actually read, and calling 
for the reader to answer questions of the type job incumbents had in mind 
when they used the materials. Tlie batter}^- will then be sc:led on a 
sample of recruit personnel of known reading ability but not possessing 
job-specific skills and knowledges. Once the reading ability re4uired 
for each of the tasks is determined, they will be submitted to job incum- 
bents for judgments of the criticality of each and the fr';;quency with 
which each is perform.^d. The final product will be an inventory of 
various types of tasks and the reading level associated wi^h each. The 
inventory can tnen be used for establishing general readi:*;. ro.nuircaents ^ 
for many Navy training or job situations.- In short, the inventory will 
make it possible to determine the nature. of reading tasks in a rating; 
the criticality of each, the frequency with which it is pe;:formed, and an 
estimate of the reading ability required in order to perforrr. it. 

It can be seen that, in one sense, this is an alternative^ -to the u,je 
of readability formulas in assessing the difficulty of mat^^rials ues-gned 
for a specific job. It has the advantage, however, of dcteiinlning reading 
abilities required for specific materials known to be used ou the' job . 
whex-GHS usir^ formulas, some (or perhaps much) of the material sampled is 
not actually used by job incumbents (such as "theory of oper:itiua*' .'or 
example). Put another way, the fact that written materialfi is "designed" 
for a particular job does not necessarily mean that it is required to 
perform that job. Needless to say, the NRTT method does not supplant 
• the need for formulas, judgments, or clo^ie tests. Wi^en any nev/ a^at-' ■"ial 
is sent into the field, it is obviously necessary to ensure, as far .iS 
possible, that the material will be satisfactory in terms of readability 
and comprehensibility . Whether it will be used^ these tests will not say. 
The ultimate test of its useability must come from the man on the job. , 

In addition to providing information as to the readincr. ability 
required to perform particular jobs, it is hoped nhat information reg. 'ding 
"readership^' u±ll also, emerge. That is, does a gap between reading abilxty 
and difficulty of required job reading materials influence the extent to 
which men report using those materials. In discussing a similar study for 
the Army,,Sticht (1975c) reports that . . the more able the reader, the 
greater the reported use of printed materials" (p. 52). 

Summary 

Summarizing the area of prediction of readability and comprehensibi Mty 
it seems clear that not just one, but a number of approaches r.hould he 
used to promote high, quality written materials. Readabil^ ry basic 
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characteristic and, in its operational sense, must be assessed according 
to a formula. But i'h'.s is not sufficient. The comprehensibility of the 
material mist be assured by means of cloze tests or human judgments, if 
possible. vThich forn^.ila should be used to predict readability, and which 
techniqae(s) should be used to assess comprehensibility are still matters 
open to question. 

PRODUCTION OF READABLE WRITING 

General 

Fredicciag the difficulty of written material "after the f act'V is a 
major problem, buc equally or more important is the problem of producing 
"readable writir;g" in the first place. For this latter task there is no 
"formula" which one can directly apply. Writing is an ART as well as a 
SKILL. Wliile it is probably impossible to tell someone how to "do art" 
there should be some techniques by which the writer's skill can be 
enhanced. This is the concern of this section, and in general, can be 
conceptualised as a three-part problem: (1) Can a writing style guide 
be devclopea which will serve as a "job performance aid" for good writing? • 
(2) Given t'r.at a writer is both skilled and artistic in his work, can 
Military Sp<f^cif ications be provided hira so that his product conforms to 
cur needs? and (3) How can we best provide the writer with "tools" to 
ensure that specified standards are being met? 

Readability formulas play a part in the production process > hut as 
pointed out so clearly by Gunning (196^"), "...formulas are tools, not 
r'lles . . .x>/arning systems, not formulas tor writing." Production is not 
simply the opposite side of the coin from prediction, although the two 
processes are inextricably related. Tlie two processes can be thought of 
as it«2rative; one writes, then assesses the difficulty of the writing, 
uhen rewrites, then reassesses, etc. The interaction between the two 
will be examined more thoroughly below in the contexts of specific writing 
aids. 

Research in the area of readable writing does give some cause for 
opcimism, but for caution as well. There have been a fairly large number 
of .-studies which have shown clearly that experimental manipulation of 
writing variables can make a significant difference in comprehensibility. 
In general, these experimenters (many of them conducting PhD dissertation 
research) have taken original passages and written easier and/or harder, 
versions as indexed by readability formula scores. In exair.ining these 
sources j the problem is one of specifying precisely iOkcut v/as changed to make 
the readability different. Most have reported that they modified word 
and sentence difficulty and little else, presumably because these are the 
two variables common to most readability formulas. Most writers would 
argue, however, that such changes are either not sufficient or are too 
simple-minded for such a complex task. It is here that a x^ord of caution 
is in order. There seems to be little doubt that mechanical shortening 
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of vords and sentences will do little, if anything, other than improve 
readability formula scores, with scant chance of improving the reader's 
.comprehension. At the extreme, it is clear that a writer with ulterior 
motives could certainly produce writing that gets a better rviadabilicy 
score and yet be less comprehensible than the original version. It seems 
clear, then, that in the research studies mentioned above the modification 
of surface variables (i.e., word length, sentence length, etc) have 
incidentally resulted also in changes to deeper (or casual) variab''es. 
This is a research question which is virtually untouched and yet WtVlch is 
critical to the understanding of how to make writing more comprehensible* 

Style Guides 

The first facet of the problem of readable writing as mentioned above- 
3tyle guides to aid the writer — would profit immensely from the research 
:,\ist suggested. The guidelines put forth by existing style manuals are 
based largely on intuition, with little, if any, empirical base- It may 
be that such manuals are of no help to writers- They have existed in 
profusion in our libraries for years, and yet we are still suffering from 
inferior written materials- One must conclude from this that such guides 
are either not used by writr^rs of military materials or that the informa- 
tion they contain is inadequate to the task of telling one "how to write." 
Information provided this author by Dr- G. R. Klare (personal communica- 
tion) indicates that the problem lies, at least in part, with the latter 
of these possibilities. Klare reports that in reviewing 15 source books 
(10 v7rittea specifically for technical vnriters) the a[;rcement among authors 
as to specific suggestions was quite low. The suggestion "use short: words" 
(for example) , which one might expect all authors to agree upon, was men- 
tioned in only tiDO of the 15 books sampled. Outright disagreement was 
found for such alternatives as "be concise" versus "be complete," and 
"keep paragraphs^ short" versus "vary paragraph length." In view of this 
apparent uncertainty among "experts" on writing, it is not surprising 
that much of our material is unacceptably difficult- It is felt that 
progress is being vudb in this area with the publication of style guides 
specifically for the military writer. Under contract with the Naval Sea 
Systems Command, for example, BioTechnology , Incorporated produced a 
guide titled "Requirements and Criteria for Improving Reading Comprehension 
of Technical Jlanuals" (Post & Price, 1974). This guide contains 17 "tests" 
for improving the quality of technical vrriting in the three areas of 
Organization, Technical Communication, and Readability. The seven te^Ls 
dealinc^ specifically with readability and comprehensibility , in abbreviated 
form, are as follows:^ 



This manual assumes that the writer is aiming at the ninth-grade level; 
this should be considered in interpreting the tests. An answer of "yes" 
to any question indicates adequate readability (at ninth grade) on that 
test, 
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1. Heading Review . Do approximately one-half of the subparagraphs 
have headings? Is material within paragraphs consistent with its heading? 

2. Topic Senteace Check . Is the heading clear? Does the heading 
cover about three or four topic sentences or key points? 

3. Words Per Paragraph Count . Do paragraphs average no more than 
45 to 60 words? Are key points highlighted if the paragraph must be 
longer? 

4. Words Per Sentence Count . Do tihe sentences average 20 words or' 
fewer? Have compound sentences and complex sentences been avoided? 

5. Syllables Per Word Count . Does the material average about 11/2 
syllables per word? Have short words been used whenever possible?^ 

6. Equipment Nomenclature Count * Is any unfamiliar nomenclature 
either defined in the text or called out on an accompanying pictorial? 

7. Layout Review . Has double-column format been used? Is each 
graphic contiguous with the text in which it. is discussed or referenced? 

It is believed that this manual (which also contains specific guide- 
lines for correcting deficiencies, a large number of "rewrite practices," 
and generous illustrations of "good" and "bad" material) can be of valuable 
assistance to the technical writer. However, it is vital that it be 
experimentally tested to verify this assumption. Work is currently under- 
way which involved the use of this guide in rewriting a technical manual 
and comparing the rewritten version with the original in terms of its 
effectiveness . 

Military Specifications 

A second major area of concern in the production of readable writing 
is that of communicating the needs of the user to the technical writer. 
This is a question of the clarity, conciseness, and completeness of 
Military Specifications (Mil-Specs) and Military Standards (Mil-Stds). 
With regard to readability and comprehens.ibility , it is the opinion of 
this writer that present Mil-Specs and Mil-Stds offer little useful 
guidance for the writer. Other matters aside, it is clearly apparent 
that no test of readability or comprehensibility' has been applied to 
these documents themselves. One sentence picked more or less at random 
from one basic Mil--Spec contained 47 words of which 21 words were composed 



The authors suggest that since manuals deal with technical terms which 
cannot be eliminated, these terms should not be included in the count 
of syllables. This procedure should, however, be experimentally tested. 
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of three or more syllables. 0-:her sirailar examples abound. .Hut aore 
specifically, with regard to s-ptr.c If icaLion o\: rtiadabillty nad comprehend . 
sibility standards, most or ail such docLiments are woefully inadequate. 
One, for example, states that *'As a general guide, the level of writing 
should be for a high school graduate having specialized training as a 
technician in military training courses/''' Even those of us in the field 
of readability/cotnpreh-ansibilit;^^ (much less the engineer/writer) ■ don' t 
really know what a typical high school graduate "looks like." And taken . 
literally, manuals written to this. prescripticu (if it were possible) 
would automatically be excluded from use in at least soiiie training courses, 
because the trainee would not yet have the "specialized training" required 
to understand the manual. It would he. comparable to zhe employment adver^ 
tisements which sperify "only experienced need apply," when the "experience 
can be gained only throu;^!! the employment* 

Another, more basic Mil-Spec gives somewhat more explici'^: directions. 
It states: "Narracive text (those pages that consist of not i.e.ss than 200 
words in con::;ecucl'/e oer.tcn-o- per pa.^,e) shall conform to the following 
readability standards: The average sequence length (ASL) shall not 
exceed 20 words .... The average word length shall not exceed 1.60 syllables,. 
The percent ^ .^rrjoi^al sit-ice-c^s (FPS) shall noi: be less ti'-an IjZ oz the 
total. ^ rne particular fl^y^jres specified are reasonable; if .such stand- 
ards were achieved, ihe r^.iadobillty .torrr.u.la score would be at approximately 
the 9th or J.Oth gradv level. i3ut, one might vronder if some writers read 
any further than whe^-e it say..; "...200 words la consecutive sentences..." 
Certain persons resnonfiLble for producing teciiuical manuals have been 
heard to say rhac "by definition" their v;riting never lias more than 200 
words per page in consecutive sentences. Tliis not to imply that producers 
or writers of manuals intentionally write poorly. Quite tdc contrary, one 
suspects that., they take pride in their work just as any professional does. 
But the fact remains that much of our technical writing is too difficult 
■for many of thie intended users. More specific guidance, and perhaps more 
.rigid quality control, would appear to be necessary conditions f^-- rectify- 
ing this problem. And accompanying such specifications, it is felt that 
we must provide the writer 'with tools to meet our standards — the third 
general problem in the production of readable writing. 



MIL-M-2410nB ^ Manuals, Technical: bunctionally Orienced Maintenance 

Manuals (FOMM) for Kquipm^jat and Syscems, Jan 74. 
g 

MIL~M-63000C (T^l) Manuals, Technical: General Requirements for Manu- 
scripts, Dec 1960. 
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Tools For Readable Writing 

^vTiat tools could be provide for writers? In general, the answer 
involves the various methods of assessing readability and comprehensibility 
discussed in the previous section. As indicated ^.^aclier, prediction and 
production go hand in hand. Figure 3 illustrates the iterative process 
that is involved. Included in this figure are some examples of the tools 
that might be provided the writer in order to produce an acceptable product. 
Basically, we see the writer producing his "first draft," relying at the 
outset primarily on intuition in predicting its difficulty. At some 
point, however, a more precise assessment of the writing should take place. 
This might take the form of a readability formula (which the writer will 
be trained to use) computed manually. As pointed out earlier, the 
formulas which are currently available, from RIDE and FORCAST to the prob- 
ably more precise Flesch (Kincaid) and Dale-Chall, are all relatively simple 
to calculate by hand. As our underi^tanding. of the problems specific to 
military writing (e-cr.^ long, but familiar technical terms) become better 
understood, and as cost-effective and more efficient methods of analyzing 
readability and .comprehensibility are developed, it would seem appropriate 
to autotnate the process of prediction to the greatest possible extent. 
Several possibilities present themselves in this regard, ordered in success- 
ively more. sophisticated techniques. 

One such technique is the Navy Automated Counter (NAC) developed for 
use in the CNETS analysis of Navy Rate Training Manuals (RTMs) (Biersner, 
1975; Bunde, 1975). To reiterate briefly, this device is composed of a 
stylus, which, when pressed to a working surface, trips a micro-switch, 
which in turn activates a counter. Use«. for counts of syllables, words, 
and- sentences for the Fle.sch and Kincaid formulas, it would be equally 
as useful for computing virtually any other formula. This could be a 
cost-effective aid which writers would probably find more acceptable than 
using simply paper, pencil, and fingers. This "acceptability" would 
probably give great^^r assurance that accurate counts are in fact being 
raade in the field. Bunde (1975), in a test of this device, found that 
it reduced the time to compute the Fl?:3ch formula by 46% over manual 
counts, with equivalent reliability. From the point of view of this 
v/riter, such a device might be considered as an inter-im technique. It is 
recommended, however, that more sophisticated procedures be planned for 
future use. 

At a somewhat higher level of sophistication are those readability 
analysis techniques which utilize a specially niodified electric typewriter. 
These are the Automated Readability Index (ARI) and the Reading Ease 
Assessment Device (READ). Given material to be analyzed during any typing 
stage of the production process, these devices (which are s^'milar in 
design) allow the writer to determine the difficulty of his writing as it 
is typed. Essentially, these devices take account of the number of 
"strokes" of the machine to count th'Ci number of letters per word and the 
number of words per sentence to be entered into a modified Flesch RE for- 
mula. They both require typing skill, however, which may be a drawback 
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for some writers. In addition, their cost may be prohibitivs (although 
not exorbitant) for situating-at the many locales at which manuals are 
produced. If however, the cost is warranted, and if skilled personnel are 
available for their use, they represent a step advanced from the" manual 
counting device discussed above. But in addition to the c.bove factors, 
these devices are limited in that they provide (PnZy readability scores. 
As will be seen below.» improvement of technical writing overall actually 
demands much more. 

At a higher level of potential, let v:5 consider two possible systems- 
one marketed by Scientific Time-Sharing, Incorporated, and the other 
developed at Harvard University.^ 

The first of these 'systems incorporates a modified RE computation, 
in which a sample of material can be input to a computer terminal which 
is connected via telephone line to a central computer. Material is typed 
at a remote terminal and, on sigTial, an almost instantaneous analysis of 
the writing is printed out. A seemingly advantageous characteristic of 
this technique is its ability to reduce the impact of long technical terms 
on the analysis. When inputting the material, if a familiar but long 
technical term is encountered, it is counted as any other word at its 
first appearance — both as a discrete word and as a polysyllabic word. 
When subsequently encountered, 'it can be enclosed in brackets which 
excludes that occurrence from the overall "analysis . The effectiveness of 
this technique, however, depends entirely on the ability of the typist 
to recognize those words which are of such interest and bracket them ±fi 
•an efficient manner* Unlike the ARI or READ devices, however, it is 
considered that the time-sharing system offers the potential for suv-h 
features as storing of technical terms in a lexicon and future automatic 
exclusion of these terms from the analysis. It is considered that future 
R&D effort might well include the investigation of such potential. 

The second of these computerized techniques is referred to as a 
^Computer Aided R.evi-sing, JEditing, and Translating system (CARET I), and 
Was developed by Kla.'e, Rc5We, ^t'. John and Stolurow (1969), Like the 
time-sharing technique discussed above, CARET I also provides readability 
analyses after the material to be assessed has been input. It .is felt 
that CARET I has at least two distinct iimnediate advantages over the 
former system. First, it provides not one, but five separate analyses, 
including the Flesch, the Fog Index, and the Farr-Jenkins-Paterson 
revision of the Flesch formula. Secondly, it provides a triple-spaced 



9 . , 
No bias in tavor of these specific developers is intended. Other similar 

(and perhaps better) systems may well exist and not have come to thfi 

attention of this author. ' ^ 
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printout of the inputted material, indicating for each word the number of 
syllables, and for each sentence the number of words. Tbie editor /writer 
can therefore adjust (as necessary) word length and/or sentence length, 
entering alternatives directly on the printout. And, like the time- 
sharing system, it is interfaced with a large computer which would permit 
the compilation of a lexicon and many other possibilities to assist not 
only in prediction of difficulty, but in writing arid re\^iting. Figure 
S^shows how such a lexicon (as well as a Key~Word-in~Context file) might 
"feed back'' into the various stages of siting. To exemplify the poten- 
tial of such a system, the entire text of (for example) an .-electronics 
manual could be analyzed for the frequency of occurrence of key words at 
the same time that the niaterial is input for readability analyses Then 
not only those words in the lay language, but also technical terms, could 
be cataloged according to their frequency and this information fed back 
to writers for consideration either in rewriting or in original writing 
of new material in the same subject matter area. Optimally, it would 
appear that the best features of both the preceding systems could be 
combined to form a procedure that is efficient, effective, and cost- 
beneficial. 

An even more sop^histicated system would be one such as the Navy's 
TRUMP (Technical Review and Update of Manuals and P^ublications) system. 
This system is currently undergoing test and evaluation at the Naval Air 
Rework Facility, Jacksonville, Florida- TRUMP is technologically more 
advanced than the others mentioned , 'basically because of its Optical 
Character Recognition (OCR) and automatic photocomposition capabilities. 
This makes it possible to electronically scan material to be input, 
resulting in a reported throughput rate of potentially hundreds of times 
greater than with keypunching. While no .readability analysis for TRUMP' 
has as yet been implemented, it would seem to have the potential for 
virtually any such analysis as well as all other capabilities mentioned 
in connection with previous systems. Of particular interest for Navy 
manuals, TRUI-IP can automatically process illustrations and complex tables 
as well as running text. Upon completion of the ongoing T&E of TRUMP, 
it should be considered as a central feature of any future R&D effort in 
the area of readability and comprehensibility . 

SUMflARY 

The underlying theme of this paper has been a plea for reformulation 
of the basic rationale underlying research in readability and comprehen- 
sibi].ity.. It recommends that future research be directed at determining 
which of the many variables involved in the readability and comprehensi- ' 
bility area stand in causal relationships with the ability of persons to 
comprehend the written word. And finally it suggests that present read- 
ability formulas, with perhaps some modification, are acceptable when 
only readability is being considered, but that a means must be found to 
ensure th^at materials are also satisfactorily comprehensible. It seems 
that the time has come to orient our efforts toward irnpteme?itation of what 
is already known in the field, and to provide for the military writer the 
means to achieve an acceptable produdt considered in terms of the needs of 
the user. \ 
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COMMENTS ON THE PAPER BY CURRAN 



• George Klare 
Ohio University 

In commenting on Curran's paper, Klare first reiterated points made 
by Curran regarding three types of siting for which readability formulas 
are not useful. First, a readability formula is not -useful for predicting 
comprehension of prose that does not follow the rules of the language; e.g 
words strung randomly together or phrases such as- '*a rose is a rose is a 
rose". Secondly, a readability formula may not be a good predictor for 
some prose that doQA follow the rules of the language, but may include 
complicated embeddings or the like; e.g., "the house that Jack built". 
Thirdly, readability formulas may not well predict the comprehension of 
certain types of semi-prose; e.g., highly technical writing, mathematical 
writing, or poetry. In this case, however, speci-^l formulas have been or 
might be developed. In addition to these three problem areas for read- 
ability formulas, Klare indicated that matters or organization "smoothnes 
of prose style", and related factors limit the effectiveness of a read- 
ability formula in predicting comprehension. And, finally, readability 
formulas do not take into consideration characteristics of readers, such 
as their background, reading competence, and motivation. In short, Klare 
agreed with Curran that readability as measured by the various formulas 
is a necessary, but not sufficient condition, for comprehensibility • * 

Readability formulas are predictive tools, and one needs to know the 
conditions, under x>;hich -they predict well and where they do not predict 
comprehensibility accurately. Readability formulas use predictive ind^x 
variables, not measures of causal relationships. This fact has several 
implications. (1) It may be difficult to improve the predictive validity 
of readability formulas by such things as incorporating a technical term 
factor in the formula. In one such case, with the Dale-Chall formula, 
adding technical terms did not alter the predictive power of the formula. • 
(2) We need to continue looking for better criteria of comprehensibility 
and useability to be predicted by readability formulas. The index 
variables predictive of comprehension may change depending on the measure 
of comprehension being predicted. (3) One must be cautious in trying to 
use what is known about the readability score to produce or to monitor 
writing. In some cases it is possible to "fool" the readability formula • 
by changing the material to get a higher readability score, while leaving 
comprehensibility unchanged. Regarding implications of readability 
research for writing, Klare suggested that if one is going to rewrite 
material for readability, he or she should work on the inaterial for the 
least motivated first* Secondly, he or she should rewrite the least 
interesting of the material before other portions of the material. 
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RESEARCH ON THE 'COMPREHENSIBILITY OF 
AIR FORCE TECHNICAL K^TERIALS 



Arthur I. Siege! 
Applied Psychological Services, Inc, 

The Air Force, like any modern organization, depends to a considerable 
degree on inforniation which is transmitted through natural language encoded 
xu T-rritten fom. Often- the transmission process is ineffective or only 
partially effective. Incre.afiing the effectiveness of this written v/ord 
information transfer process is the thrust of most current Air Force pro- 
grams in the topic area of readability research. Specifically, the Air 
F.orc.e Is r.oncomed with increasing the coroprehensibility of written mate- 
rials, developing methods for diagnosing where and why written materials 
are not comprehensible, and developing automated techniques for assessing 
the comprehensibility of va-itten materials. Since technical training and 
ecTvcation represent a major aspect of the Air Force's peacetime activity, 
and since a substantial portion of the training process involves trans- 
mitting 3-nformation in x^ritten form, it is not surprlsi.ng that much of . 
this research has been performed in a technical training context. However, 
tacrn is little reason to believe that the results and implications of 
prior and current Air Force work are not mostly generalizable to other 
contexts; e.g., operations and maintenance manuals, bulletin board notices, 
and technical orders. 

Partly because of the technical training orientation and, more impor- 
tantly, because of definitional problems, much current Air Force work has 
v^^.ared away from media and foraat problems. Taere are, however, .soma 
exceptions to this generalization (e.g., Davis, 1965). The definitional 
prooxem seems to arise from confnsion among some researchers between what 
is meant by ^'utility" as opposed to the meaning of "comprehensibility." 
Job aids are judged by the criterion of utility; text is judged by the 
criterion of comprehensibility. A text may be quite comprehensible, but 
be of little use in certain operational situations. A job aid may be 
quite useful in a given operational context and be low on comprehensibility 
Similarly, such items as page format, font, and type size may have much to 
do with reading rate, but have little involvement with comprehensibility. 
X'Then it comes to encoding language through the x*rritten process, media and 
format problems are mostly related to utility rather than comprehensibility 
Cnmprehensibility and utility should not he confused. The differentiation 
is not trivial and the Air Force has tried to avoid this confusion. 

Subsequent sections of this paper attempt to recapitulate current Air 
i^orce trends relative to increasing the effectiveness with which students 
organize the knowledge encoded in the language appearing in their instruc- 
tional materials. 
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Auditor:' Supplementation * • . 

A set of recent Air Torce^ studies has iiivestigated the gain, if any, 
that can be obtained from supplementing written textual training materials 
with auditory presentations (Sellman, 1970; Siegel, Lautman, & Burkett, 
1974). In this technique, while the reader reads the textual materials, 
the material is also preoented to him through a headset- Accordingly, 
the approach has been called auditory supplementation. The advantages of 
the technique appear to be. that it might (1) serve to refccus the reader's 
attention in the case of distraction, (2) help the person who can compre- 
hend spoken language but who possesses a reading disability, and (3) 
serve as a redundant and noninterf er iiig information source. Present 
indications from this work are that auditory supplementation may be helpful 
in increasing comprehensibility in certain situations. Sellman, working 
at the Air Force Human Resources Laboratory, found that individuals using 
simplified materials plus tapes scored higher than persons using simplified 
materials alone. Siegel, Laucman, and Burkett, who also worked under Air 
Force Human Resources Laboratory sponsorship, partially confirmed Sellman's 
findings. Although macy interactive effects seem to exist, by way of 
generalisation, it seems that auditory supplementation will be helpful when 
one or several of. the following conditions exist: (1) the mental ability 
of 'the reader is low, (2) the reading grade level of the reader is low, 
(3) the materials to be used are of the discursive rather than the 'Mpok 
up" type, B.ud (4) reading materials represent the sole source of the 
information to be learned - 

Comp re h e n s i b i 1 1 ty Me a s u r em e n t 

Tlie Air Force Vs emphasis in the comprehensibility measurcroant area 
focuKes on developing new comprehensibility measurement concepts, validating 
tliese concepts, and implementing them. Accordingly, the approach consists 
of a basic research vector, an advanced development vector, and an applica- 
tion vector. Each draws from and is based on the other avenues and consid- 
erable interplay occurs. This overall approach seems to Introduce consider- 
able economy and cross-fertilization into the cycle. 

For a comprehensive review of earlier work relative to comprehensibility 
measurement, the reader should see the Air Force Human Resources Laboratory 
report authored by VJilliams, Siegel, and Burkett (1974). The conceptual 
emphasis of the present work is based on the conjecture that prior compre- 
hensibility measurement techniques (e.g., Flesch, 1951; Dale& Chall, 1948; 
and Lorge, 1944) deal principally with what one might call mechanical 
factors such as quantities of words, sentences, syllables and their 
occurrences— no t with meanings per se. They have been used for some time 
not only because they measure reading difficulty (specifically, reading 
grade level) in some sense, but also because they are suitable to rela- 
tively easy calculation by hand. However, such measures are not diagnostic. 
That is, knowing that a given text is at such-and-such reading grade level 
will tell the author little about how to improve the writing. Moreover, 
the concepts included in such mechanical measures are atheoretical . They 
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possess little foundation in language processing or cognitive theory. 

Itionally, one can use small words and sentences to produce difficult 
•rial or, alternatively (given a technical situation), one can use big 
' and sentences and produce highly comprehensible material. Finally, 

Li. .se measures have been largely validated in high school/grade school 

situations which have little relationship to adult technical training 

situations. 

Tl.ri orientation of the Air Force's present work relative to corapre- 
hensibility measurement is on the assessment of the cognitive and the 
language processing load which a text places on the reader, Alternatively, 
the measures under development seek to assess the amount of thinking a 
text forces a reader to perfonn. These measures, accordingly, represent 
an attempt to reflect and quantify what the text forces to happen inside a 
reader's head, rather than to determine comprehensibility on the basis of 
sizes and frequencies of words, length of sentences, and the like. Some 
support for the contention that cognitive processing aspects represent 
a salient aspect of comprehensibility comes from the work of Coke (undated) , 
w]io concluded as the result of a readability study in another context that 
"...reading rate is sensitive to the amount and/or type of cognitive 
processing required by a reading task." Additional support seems to 
derive from the ongoing work of Davis (1975) at the Air Force Technical 
Institute., relative to the effects of mode of expression on comprehonsibil- 

Uithin the intellective load structure, two separate, but related' 
approaches are under investigation: (1) an "intellective load" analytic 
approach,' and (2) a "language processing" approach. 

The intellective load analytic approach is based on and drara from 
the structure~of-intellect model of intellective function developed by 
Guilford and his associates (Guilford, 1967; Guilford & Hoepfner, 1971). 
This model is the result of over 20 years of work x^rhich empirically 
isolated and defined 120 intellective functions. A logical choice was 
made among the 120 Guilford functions on the basis of rational relation- 
ship with textual comprehensibility. The Guilford functions selected, and 
on which textual comprehensibility assessment techniques were developed^ 
are cogniti&n of semantic units, cognition of semantic relations, 
evaluation of symbolic implications, and divergent production of semantic 
units. Full descriptions of ^hese metrics are found in Siegel and Wolf 
(1975) and in Siegel and Bergman (1974). The Siegel and Bergman report 
also presents the results of an initial validation of these measures. 
Cross validation and predictive studies are currently being performed. 

The language processing approach is based on concepts drawn from 
current /psycholinguistics. Metrics have been developed to reflect each 
of the following: Yngve depth, morpheme volume, transformational com- 
plexity, center embedding, left branching, right branching, and complement 
deletion. These metrics are fully defined, described, and elaborated in 
Siegel and Wolf (1975) as well as in Lambert and Siegel (1974). The 
Lambert and Siegel report . also describes the methods, procedures, and 
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results, of an initial validation study relative to these psycholinguis- 
tically oriented measures. Currently, cross validation and predictive 
studies are also being perforn^ed relative to metrics. 

Automated Measurement of Comprehensibil ity 

The Air Force's goal in the automated comprehensibility measurement 
sphere is the development of an on-line technique which will handle 
the logic and calculational sequences required to determine the selected 
structure-of-intellect and psycholinguistically oriented measures describ- 
ed immediately above. The computer program which will achieve this goal 
is currently being specified and set in logic form. It is anticipated 
that formal programming will start during early months of 1976. 

Obviously, a part of the computer processing of text for publication 
i3 the "typing" or optical reading of the text. Some text editing pro- 
grams also use a computer-stored dictionary. With this capability, a 
computer can accomplish functions such as automatic hyphenization, page 
numbering, inde^cing, page layout, spelling checks, centering of headings, 
and the like. The comprehensibility measurement progran which is being 
developed could become the back end of the more routine text handling 
procedures now ^available or could stand on its own. 

Some concept of the organization of the comprehensibility measurement 
computer program can be gained from Figure 1, Input and interaction with 
the program can take place in the batch mode or through a terminal. The 
operating subroutines represent those program modules necessary for 
setting up and performing a textual analysis, Tlie semantic subroutines 
are those subroutines necessary for performing the calculations underlying ■• 
the selected structure-of-intellect and psycholinguistically oriented , 
metrics, A parsing subroutine is included because of the dependence of 
certain of these metrics j on sentence structure and syntactic attributes. 
Results will be presented, at user option, by sentence, block of sentences, 
and total text. Block and sample size are also variable under user control. 
Output will be in usual printed format, on cards, on tape, or on the term- 
inal (in the interactivej mode) . 

The utility subroutines include a dictionary update routine (needed 
because maiiy technical words may not appear in the usual computer-stored 
dictionary, a teaching ^ubrou^ine (to teach the user how to use the total 
program), and a number of statirs^^ical subroutines (to summarize data). 

Thfe support subroutines represenX those usual procedures needed for 
an effective program (checkout, alarm, etc), a subroutine to plot direc- 
tional trends, and norm.^ for comparing a text under analysis with a com- 
parison set of texts. The required norms have already been developed. 

Figure 2 further details the analytic sequence in its current form. 

The reader who is interested in an earlier Air Force approach to the 
automation of comprehensibility analysis will find the work of Smith and 
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Kincaid (1970) and that of Smith and Senter (1970) of r,elevance. Other 
work in this regard has been performed outside of the Air Force context 
(e.g., Coleman & Liau, 1975).- 

The Future 

It is interesting to conjecture about the future trends of the Air 
Force's interest in comprehensibility measurement. It seems that for the 
immediate future, the current trends will continue. The search for new 
and better diagnostic comprehensibility constructs will probably continue, 
along with ongoing validation and extension of the metrics currently being 
achieved. The rapid integration and application of tested constructs into 
the operating technical training system also seems likely to continue in 
view of the continuous, large scale, training mission of the Air Force. 

One can also anticipate extension of many of the implications of 
work, such as that which was described in the present paper, to the techni- 
cal and operational manuals situations. The Navy has already perceived 
this need and has included comprehensibility evaluation in its current Navy 
Technical Manual System planning (Sulit, 1975). The current Air Force ^work 
also seems to possess some implications for natural language processing, 
and extensions in that area, relative to lecture material, seem possible. 

Finally, the Air Force has made a start toward increasing the effec- 
tiveness of technical training textu.^.l materials, understanding the compo- 
nents of textual comprehensibility, placing these components in some ' 
quantitative form, employing them so as to allow evaluation of the compre- 
hensibility of technical training textual materials, and developing a 
diagnostic/prescriptive tool which will allow writers to produce texts 
which arc more comprehensible to students. 
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COMMENTS ON THE PAPER BY SIEGEL 

Ernst Rothkopf • 
Bell Telephone Laboratories 

As a preliainary opening comment, Rothkopf expressed his opinion " 
that the military appears to have a concern for the human use of ^/r- tt -. 
materials, and to have shouldered the responsibility for making such 
documents more effective. Rothkopf felt that civilian publishers, 
universities., and school systems have been relatively irresponsib] in 
this effort, or perhaps they have recognized that the need existed, but 
saw no means for implementing any action in that direction. The irilitary 
x.ork may provide guidance for the civilian sectors on the practical ways 
in which- people produce written materials for useful purposes. 

Regarding military work on the comprehensibility of materials, 
Rothkopf made some general remarks regarding the relevance of the military 
work to those who are engaged in more fundamental research on human 
learning and on language processes. The point made was that the work in 
job aids and other documents that are supposed to support job behavior 
reveals the folly of accepting the word "comprehension" as a primitive 
term, the way it has been accepted as a primitive term by many psycho- 
linguists and students of cognition. When simply studying, learning, 
and recalling information from text, it is possible to never know that 
one has not understood. But when the printed material is supposed to 
take one to the next stage in a procedure or sequence, one quickly ■ 
recognizes that language is not all that informative, and that understand- 
ing IS a very complicated thing. 

^ Directing his remarks to Siegel's paper, Rothkopf made two major 
po3.nts. First, he agreed with Siegel that internal measures of text 
(such as word length and sentence length) are interesting and ought to be 
explored. However, he stressed the idea that the variables counted in 
readability formulas, such as word length in the Flesch formula, have a 
certain construct validity as measures of more psychologically interesting 
variables, such as familiarity, which in turn have an effect on learning, 
retention, and so forth. Thus, word length variables are not without 
psychological meaning. The second major point concerned the comprehension 
measures which Siegel reported were planned to be used. Rothkopf suggested 
that a parsing program which took semantic considerations into account but 
which operated only on measures internal to the document would not provide 
an adequate evaluation of the document's comprehensibility. He submitted 
as an area for research the idea that it is meaningless to think about 
evaluating a document by itself; the only way that a document can be 
evaluated is with reference to some outside criteria. The outside criteria 
have to come from the_ purpose of the document or some other pragraatic 
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concern* Thus> a document characterized as being coraprehensible on the 
basis of some general measure derived from internal measurement Joay be a 
completely inadequate document for its intended purpose, because it can 
contain wrong or irrelevant information, Hothkopf cited a study in which 
25 or 30 structural variables were examined, and not a single one had the 
predictive power as the amount of irrelevant material that was in the 
text* But the irrelevant material cannot be detected without reference 
to some purpose external to the docximent. For this reason, Rothkopf saw 
the need to somehow bring the content variable into the evaluation of 
documents • 
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U.S. ARMY RESEARCH AND DEVELOPMENT ON 
READABILITY AND USEABILITY OF PRINTED MATERIALS . 

Richard P. Kern 

U.S. Army Research Insititue for. the Behavioral and Social Sciences 
.-INTRODUCTION " ' 

descrSed''i!*tt?^'^'' literaturP, research and de^relopment efforts will be 
(1) rin- . ^^^""^ ""^^ addressing thr,.e problem areas: 

til olientatlo' P '"^ literature, (2) the content and 

usLbilS OF th T ^ literature,, -and (3) assessment of the utility or 
user literature when placed in the hands of the intended 

READING DIFFICULTY LEVEL OF ARMY JOB AND TPJ\INING LITERATURE 

■of Ar^v^frr/t^''^ research which examined reading difficulty levels 
D^f.^ training literature was stimulated by the Department of 

Pr J't iToTor^''" °' ''''''' -Pl--tatXon of 

nZilrl n? ' .^^^Pected to produce an influx into the Army of large 

-i^J^^n^.1 '"''^^gi'-l °- 1°" literacy skills. The research ' 

oScSvc-s of d.' er^^-'" ''f'' "^^^ HumRRO.'had the brbad 

ODjcccivcs of developing information required to identify literacy skill 

LJ^oT'tS" °'t%™'^K/'^' ^^"^^ ^-^^ - this infot- ' 

the Irmv ^o IT -PProaches could then be developed to enable 

Too nnn -^"^f ^vely train and utilize the lower aptitude Project 

iSe'Ar° P^^^°nnal. The resulting program of research carried out for ' 
the Army under HumRRO Work Units REALISTIC, READNEED, and FLIT is 
sunumarized in Caylor's paper in this volume. Becaus^ of the ea^^y 
and tr^r- ^^l-^^^"^^ °f these studies to Army R&D on -job 

relTpr;i"''f r^'"""'"""' ^""^"^'^^ °^ ^^e readability a^d 

readership findings will be summarized here. ) 



Reading Sk-m Level of Personnel and Reading Difficulty Level of Manuals 

m-,.c^^^^^':''^^''^^'^ that one obvious point of difficulty for the 
marginally literate personnel would be the Army literature thev were 
expected to use in learning and performing their job. Research (Sticht, 
bents ^ith Jh; ^°"^P^^i^^f,^f^i"g -kill levels of low aptitude job incum- 
this exne.ta^ difficulty level of their job literature confirmed 

^he h?:^r ^ However, this research also indicated that many of 

the high aptitude job incumbents could be expected to have difficulty in 
using the same literature. 

P^ n/'^^";'o?^^ T^f "d^^'^d' fr*^™ the Work Jnit REALISTIC report (Sticht, 
?o . Vrpr r ^"^^^f^^- that reading and learning from Army literature 
manv Mf^''"^', aptitude personnel but., also, for 
many high aptitude personnel. This figure presents a visual comparison 
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between Ch- average reading difficulty level (modified Flesch formula) 
of manuals c' ted by incumbents and supervisors as being used on the job, 
and Che average reading skiir levels of incumbents in three high density 
Army jobs. Note tha: for repairman and supply specialists the average 
ditticulty level ot the manuals exceeds the average reading skill levels 
of even th..-. high aptitude incumbents by 5 to 6 grade levels. The reading 
materials used by the cooks, however, appear to be reasonably geared to 
the reading skill levels of the high aptitude cooks and, while difficult,' 
possibly still within the range of the low aptitude cooks. The lower 
difficulty levels of the cooks materials reflect the fact that the bulk 
of the materials cooks reported using on the job were their job aids 
the recipe cards. ' 

A second, related set of results from Work Unit READNEED are 
presented in Figure 2 (Sticht, et al., 1973). In this case the manuals 
studied were identified from the list of study materials prescribed for 
the soldier a use in preparing for his annual Primary MOS Enlisted 
Evaluation Test in each of the seven MOSs. The FORCAST formula, developed 
m Work Unit RE.VDNEED (Caylor, et al., 1973) for the Army popt;lItion ' 
and Army technical materials, was used in estimating reading difficulty 
level of these manuals. Average reading skill level is indicated by the 
verjxcal lines for three groups of Army personnel: Army Preparatory 
iratning (APT) graduates for FYs 1968, 1969, and 1970; Category fV job 
mciimbents; and non-Category IV job incumbents tested during this .-ud a 
related. research program (Vineberg, Sticht, Taylor & Caylor, 1971). the 
A, T program, at the time this research was conducted, contained remc-diPl 
reading instruction in the context of a sax-week general educational 
development program. Use of the study materials to prepare for their MOS 
r '^''''^ "'""^'^ clearly be a formidable if not impossible task for 

the APT graduates and the Category IV personnel regardless of which of 
these seven MOS they might be assigned. It is equally important to note, 
however, that these study references are written at such a high levelof 
difficulty that even the higher aptitude (non-Category IV) personnel will 
De expected to have problems in reading and learning from them. 

Reading Skill Level '.of Personnel and Use of Manuals on the Job 

Studies conducted in the fields of journalism and advertising would 
lead us to expect that as the reading difficulty level of material 
increases, readership or use of the material would dec-ease. From this 
we might extrapolate and assume that job incumbents with, for example 
lOth-grade reading skill levels, will be more likely to use job manuals 
written at, say,, a 12th-grade reading difficulty level than will job 
incumbents with lower reading skill levels. 

Estimates of the extent to which incumbents of differing reading 
skill levels used manuals or other printed material in carrying out 
their job duties are reproduced from a Work Unit REALISTIC report in 
Figure 3 (Sticht, et al., 1973). These estimates were obtained during 
interviews conducted with job inc^ymb en t s . Using a version of F]anapan's 
Critical Junidcnt technique (J^rTnagan, 1954), each incur.ibent was asked 
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Figure 3. Reading Ability and Use of Job Reading Materials 
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to give five instances during the past month when he had consulted printed 
materials in connection with carrying out his job. Extent of useage is 
plotted m Figure 3 as percent of maximum possible number of citations- a 
group could give. For example, if each man in a group had given the 
maximum of five instances when he had .consulted printed materials, this 
intormation would have been plotted as 100%; if the members of a group 
Jiad given an average of 4 citations each, the group average would have 
been plotted as 80% of the maximum possible, and so forth. 

As shown in Figure 3, extent of useage of printed materials is 
related to reading skill level of incumbents for supplymen and mechanics. 
^ooKS show a much higher extent of useage and no apparent relationship 
between extent of useage and reading skill level of incumbents. 

The estimated average reading difficulty levels for the job materials 
cited by incumbents in each of these three jobs are those presented 
earlier in Figure 1. These difficulty levels, plus the fact that the 
bnXk, of the materials identified by the cooks was recipe cards, suggest 
soma of the factors tliat must be considered in studying incumbents' 
useage of printed materials on the job. Cooks generally reported using 
recipe cards- to retrieve information on quantities of ingredients while 
preparing a particular dish. In addition, they frequently reported that 
tuey were req;-.lred by their supervisor to have the correct recipe card ' 
in tront of them wh.Q.j preparing whatever food they had been assigned. 
These observatiouEj suygost chat when given job aids designed to fit the 
tasks the man performs and a supervisor who requires that the man have 
these aids at hand, extent of useage of these materials on the job will 
be high. In contrast, neither- mechanics nor supplymen had j.^b aids 
comparable to the cooks' recipe cards. In addition, neither of these' 
latter two groups were required to havy references at their fingertips 
while performing their various tasks. vrnile extent of useage of printed 
materials is related to reading skill level of these incumbents, use of 
printed materials is relatively infrequent even among the better" readers. 
Observations m this case suggest that given job aids designed as general 
references, .bitten at a high level of difficulty, and no requirement by 
the supervisor that they be used, then, extent of useage on the job will 
remain relatively low even for the proficient readers. 

Use of Manuals on the Job and Quality of Task Performance 

. The/ research reyi'ewed in the preceding section indicated that among 
repairmen and supplyriipn,' use. of manuals on the job was positively- 
related to reading skill level of the incumbents. However, is there any 
evidence that use p-f m'anuals improves the qu.ality of task performance? 
Data related to tfiis question were obtained in Work Unit REALISTIC. 
Stlcht, et al. (1971) describes this research as follows: 

In that resf'iarch, job Incumbents were a'dminifitered three- to 
five-hour Job Sample tests in which Repairmen actually repaired 
vehicles and Supply Clerks- worked in simulated offices filling 
out forms and counting equipment. In these Job Sample, tests, 
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job manuals were available so that men who were being tested could 
use the manual if they wanted. • Figure '4 shows relationships 
between reading ability,- use or non-use of manuals, and performance 
on the Job Sample tests for Supply Specialists and Repairmen. It 
is apparent that men who use manuals did better than those who did 
not use -nanuals regardless of ' reading ability. Furthermore, of 
those who used manuals, performance was better for the more highly 
literate men. \ 

Initial Approaches to the Readability Problem by Army Commands 

In view of the research findings on reading difficulty levels of 
Army training literature and average reading skill levels of enlisted 
personnel, Headquarters, United States Continental Array Command (CONARC) ' 
established the 9th-grade reading difficulty level as a maximum difficulty 
level goal to be used in preparing and reviewing literature intended for 
entry level enlisted personn-l. In addition, CONARC directed that a 
review be made of the reading difficulty level of all manuals produced by 
CONARC schools. The 20 CONARC service schools, using the FORCAST read- 
abilitly formula, calculated average reading difficulty levels on each of 
A/0 Field Manuals (FMs) and Technical Manuals (TOs) for v/hich they had 
responsibility. These analyses revealed that approximately 94% of the 
CONiVRC irHauais exceeded the 9th-grade reading difficulty level and, in 
fact, approximately 63/j had an average difficulty level of llth-grade or 
higher. A frequency distribution showing the number of manuals at each 
reading grade difficulty level is shox^m in Figure 5 (CONARC letter dated 
29 Jan 1973). 

In reviewing these results CONARC recognized that not all of their 
470 manuals were either designed for or required by enlisted personnel at 
the entry skill levels. However, examination of the readability of their 
literature highlighted the fact that literature management policies in 
effect at that time did not require specific identification of job, rank, 
and skill level characteristics of the population for whom a given manual 
was primarily or exclusively intended. Thus, readability analyses focused 
attention on the management of literature production and the need for 
designing literature to meet the needs and abilities of specifically 
identified groups of users. Research initiated by the Army Research 
Institute (ARI) to assist CONARC in providing guidance and assistance to 
their literature management and production agencies is described in the 
next section of this paper. 

The Army Materiel Command (AMC) initiated studies to develop read- 
ability standards for use in guiding design and preparation of the many ' 
hardware operation and maintenance manuals they are responsible . for 
procuring. This work (Army Materiel Command, 1970), initiated before 
the development of the FORCAST formula, resulted in the following recom- 
nended readability standards: ' 

1, Average sentence length in narrative text should not exceed 17 
words. 
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2, Average vrord length should not exceed 1.5 syllables per word, 
excluding technical nomenclature (mandatory) words. 

3« At least 20% of the sentences in the narrative t£^xt: should be 
personal sentences. 

In establishing reasonable standards for equipment manual \7riters, 
AMC staff were concerned about the effect of technical nomenclature on 
the resulting estimates of reading difficulty level. These words 
generally consist of a number of syllables and, according to estimates 
made by A>IC staff, constitiated about 1/3 of the words used in their ! 
equipment manuals. On the premise that the user already has or quickly 
acquires familiarity with these words , AMC decided to exclude them in 
computing the average numtJer of syllables (average word length) per 100 
word sample. . Using three different formulas (Flesch, Gunning, and Army- 
Dale) they reported computing reading difficulty estimates on a number 
of equipment manuals using both the standard method of computation and 
their proposed method of excluding manadatory words. They report that 
exclusion of the mandatory words resulted in a fairly regular drop of 3 
to A read Lng grade levels as compared to estimates which included these 
words. Thus, an equipment manual which is estimated to have a 12th-grade 
rending difficulty level based on standard application of the Plesch 
formula, drops to an estimated 8th-gradc reading difficulty level by simply 
excluding the mandatory \<fords and substituting additional, consecutively 
occurring, non-mandatory words. 

There arCj of course, differences of opinion about the appropriateness 
of this approach. Perhaps the weakest aspect of this approach is the 
assumption that the user knows, or quickly learns, the technical nomen- 
clature words. Studies of: j ob-cxper.ienced and new, or inexperienced, 
incumbents' familiarity with technical nomenclature, if conducted in the 
Army, have not been rc^ported. One such study conducted in the context of 
Air Force helicopter maintenance, suggests that the only job incumbents 
who have a high degree of familiarity with the technical nomenclature 
used in m.anuals are the manufacturer's enpineers (Jablonski, 1971). 

USER ORIENTATION AND CONTENT OF ARMY TRAINING LITERATURE 

In Mcjy 1973, the Army Research Institute initiated research, under 
contract with HuniRRO, to identify user problems with existing literature 
and user requirements for revised training literature. In addition, 
research was al.so initiated under this same contract to identify and 
develop guidance to assist Army writers in preparing literature appropriate 
to the user's job needs and reading skill level. 

Survey of User Requirements for Army Training Literature* 

The survey of user requirements for training literature was conducted' 
within active Army units by means of a scrr.ctured interview (Showel and 
Lrennan, 1976) ». Personnel surveyed included students, ii^struc tors , and 
training managers a^3signed to Army Training Centers a^id Service Schools. 
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In addition, interviews were conducted with first line supervisors 
responsible for conducting instruction at the Company or Battery level in 
Infantry Divisions, and those' responsible for fraining mana^einent at the 
Company, Battalion, and Brigade levels. 

Findings applicable to the general body oi: training literature 
include complaints that information contained in FMs and TMs is frequently 
out of date, scattered through several different manuals, and presented 
at such a high level of generality that it is difficult to determine .how 
it applies to specific situations. In addition, students reported that 
the literature was difficult to read, dull, too wordy, and that it was 
difficult to locate needed information in the manuals/ 

In general then, these interviews indicated thc-.t reading difficulty 
level* of the literature, while recognized as a problem, was not considered', 
as gceat a problem as the highly generalized orientation of the content 
and the failure to organize and bring together content relevant to the 
user s* needs into one compact, easil}' accessible source. 

Development of Guidance for Army Writers- 

In addition to the survey of users of Army training literature, 
-researcher conducted interviews with Army writers and editors. The 
purpose of these inL-erviews v/as to find out what typf. of traininr; or 
experience writers brought to their job,, what type of guidance and 
support they received, how they perceived the purpose or the function of 
the literature they prepared, and what they thought a writer's guidebook 
should include to be of help to them (Kern & Sticht, 1974), 

Inforr^ation obtained during these interviews indicated that vn: iters 
were generviUy selected on tlic basis of their previous or current xfor- 
mance as classroom instructors. In many cases the vnriting assignment v;as 
added duty for an instructor. 

In general, the purpose or function of the training literature, from 
the writer's perspective, was to provide official, detailed, reference 
support for the highly generalized lecrnre-training approach that was 
still prevalent in the schools at that time. Writers vzere provided with 
little or no instruction regarding users, purpose-, or scope when" tasked 
to write a new manual. Guidance generally was simply to write a manual 
on topic "X'\ Consistent with this guidance, writers viewed their job 
as that cf assembling, organizing, and recording all that was known 
about a topic. They assumed that if a manual contained a reasonably 
complete exposition of the topic, it could serve as a. general text or 
reference source and be used by anyone interested in the topic regardless 
of the specific nature of the reader's job duties or information needs. 

Most of the writers interviewed said they did not use writer's guides 
of any type. Guidance regarding format and organization was obtained by 
usijig previous editions of the same manual or a related manual as a model 
and preparing the new literature to look like this model. V/lienever 
possible they cut and pasted from the earlier manuals as well as from 
current editions of related manuals. Kditorial review of the writer's 
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draft was generally limited to grammar, proof reading, and confonnance to 
format policies. Readability formulas or other methods for testing the 
difficulty level of the vriti-ng were not being used. 

The above observations indicated that if writers were to produce 
literature to support the Array's emerging demands for performance- 
oriented training, they needed help in re-orienting their perception of 
the purpose and functions of training literature. They would also need 
guidance in how to research, develop, and prepare this literature so 
that it was appropriate to the job-related information needs and reading 
abilities of their targeted users. It was recognized that a guidebook 
could not be expected to accomplish these objectives unaided. It was 
thought, however, that it could provide a basis for further training 
efforts by literature management » 

(Ker'.t, Sticht, Welty & Hauke, 1976) was designed to capitalize on the 
■v7ril:ers' reported tendency to use existing literature as a model for 
writing style, organization, and format rather than devoting his time to 
the study of technical writer's guides. It seeks to acquaint writers with 
the user orientation of performance-oriented writing by presenting con- 
trasting examples of topic-oriented and performance-oriented writing. It 
also presents detailed guidance on how to identify a primary user for a 
proposed manual and how to proceed in developing, organizing , and writing 
the material to meet this users' information needs. 

The Guidebook deals with the difficulty level of writing in the 
following ways: 

1. Presenting short reading tests which the writer can use to 
compare his reading level with the average reading levels of different 
groups of enlisted job incumbents. 

2. Presenting many examples of difficult witing ("Before" examples) 
obtained from a wide variety of Array manuals. These "Before" examples 
are accompanied by a contrasting version ("After" example) rewritten to 
reduce reading difficulty level and to shift the orientation of the 
material from a topic-orientation to a performance-orientation. 

3. Providing directions on the use of the FORCAST formula to check 
drafts for possible rewrite needs. 

In addition to the guidance described above, illustrated guidance 
is provided on identifying when and how illustrations can be helpful and 
on design of tables of content and indexes to make it easier for users 
to locate information in a manual. 

This guidebook, using an unconventional approach for a writer's guide 
has been well received by Army writers. It is currently undergoing test- 
use and review by the U.S. Army Training and Doctrine Command (TRADOC) . 
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Development of User-Oriented Literature by the Army 

of user-oriented IJteratur^ S^tI VT initiated development 

(USACATB) served al l Tallll., , ^""'^^^ Training Board 

TRADOC and also perf^H ^^ ^ ^°°^dxnator of these efforts for 

of .he early'juhl'icf .^1^: ela^'w p^ubl'^^^'^^ 

. Individual combat sold^Pv publications, aimed at the 

skills the soTdLr would ;e:d to f "^'^'^ °" ^-'^^^^^^ 

duty; for example hov to dpt ^ relatively specific conibat 

covor, and conceaime:i '°tL EpTbr'Sf^ °J '° camouflage, 

"era depicted as narrators Pf for brief and colored cartoon figures 
attention included, in addition to thf f individual soldier's 

such a. "To Catch a Taut l±Tc ° S cartoons and color, titles 

contained a post-paid rtlvcfrd ' ^^^^ publication 

' back regarding hir^encM^^ card requesting the soldier to provide feed- 
this type of lublLatio: Sh^^. Publication: whether or not he liked 
and, whether or rot L SouS iS T information useful; 

-anner. Informal reports by USACaS ° ''^^ '"'^""^ Presented in this 

Personnel\hf c:-id-d^rru-d-f- ^ 
mnnber'orn^w'eauJn-^"?' '° user-oriented literature a 

combined^fSr^roT-s ~g (S;:.ic)''^i ^^^^"^^ 

These manuals were desLS S^f-flT^ J ^"'^ '^'^ equipment (AMC) Commands. 

present content by lelnl ff iHus"ratio°ns M °Perator's needs; to 

supporting text; ^o avoid referencJn. f integrated with concise, 

emphasize warnlx^s by use of f ! T °T ^^^^ another; and to 

-Ln^,^. Dy use of a second color (Braddock, 1975) 

weapoSrhartr:ditio"na\'i "-^-^11 procedures for crew-served 

described the\^:f i ^.^^^^^^^^^^^^^^^^ - - ™ while a separate I.I 

have recently completed and ' "^^^ntenance. AJIC. and TRADOC 

manual contaLing all of th. 'nfoS^''"' '° '^"^^ " ^^"^^^ operator's 

-intain a given^rew-^frCel ;SoTSraddocT^L75^ " ' 

tradilt\:^L\L^!o:;'n^:d'Mt:raT'" °h ^^^"^^^ ^ --^^e 

oriented literature detelnn ff ^'''^ Prepare It as pcrf ormance- 

in training as- S as ?h1 lo^b':'^' "-ds while 

ASSESSIfJG USEABILITY OF ARMY LITERATURE 

the A^^Vs'^tfeSio" o'n\f 1 .^'^'^ ^^P^^' ^^^^ — focused 
SubsequLt researcr-^nd ''^^^^.^^^l^\^^''---^^y level of its manuals. 

initial focus trinc^ude he'^rob^ ^^^,"^^^2^-- ^ ^-^dened this 

Jnciude Lhe problems of content and the nbsence of a 
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user-orlentaLion in the design and presentation of the information. 
Now that variouo efforts are being made to develop literature intended 
to sunnount these problems, £he Army's attention is' being further 
extended to focus on the problems of verifying the useability of new 
manuals once produced and placed in the hands of the user. 

The Army's long standing practice of sending nex<r manuals to approp- 
riate schools and field commands for review and comments cannot provide 
the type of information needed to verify useability in the hands of the 
intended user (Highlander, 1975, Kern & Sticht, 1974). In looking for 
an effective approach to this problem, Smith (Smith, H. L, , 1975) has 
proposed, adopting the conceptual approach used for developing and verify- 
ing the Array's Training Extension Course (TEC) materials. 

TEC represents the Army's new approach Jgor developing multi-media, 
performance-oriented, self-paced instruction which can 'be e^-:ported to 
Array field units. These courses have utilized audio/visual, audio only, 
and printed (programmed- text) media. The audio /visual version of TEC 
IS considered the "workhorse" of the program and may be Tised either on 
an individual basis or a small group basis. The developmental approach 
used in TEC (Smith, T,., 1975) consists of first identifying ^critical 
tasks to be trained; secondly, analyzing each task to deterrvine optimum 
procedures and to identify performance standards; thirdly, usin^' the 
-performance standards as a basis, developing;' '■"crfonraneo tes^-r,;^and 
fourthly, dev-loping training to teach the r rTnanr.e test. A m^jjor 

feature of rv.' v::lopiuvn-ital cycle is the repair eiiu^nt r.ha^ dralit lesson 
materials be . >j:-ct.-d to user testing to determine their effectiveness 
.•n communicating to the intcnd-d soldier population. Soldiers' responses 
to selE-study questions contained in the lessons, their performance on 
pre-^and post-tests, and their responses obtained during interviews are 
studied to identify the need for revision of the materials. If revised, 
macerialG are again subjected to user testing. This cycle continues 
until the materials meet previously specified criteria for acceptance. 

Smith (1975) reports that applying this same approach to development 
of Technical Manuals will resul-t in manuals which present otherwise com- 
plex information in the fop of step-by-stop instructions. Furthermore, 
by using this approach- to concurrentlydevelop training and Technical 
Manuals, it is believed that, fewer tasks will be selected for traininc^ 



It should be noted that researchers dealing with the design of job and 
training literature to support equipment maintenance .personnel had 
been focusing on the problems of identifying content requirements and 
developing user -oriented literature for a number of years prior to 
research reported in this paper. Fink (1967) presents an excellant 
summary of conclusions, relevant to the issues, based on research 
conducted for the Array up to 1967. 



226 



218 



since they can be adequately covered by step-by--step instructions presented 
in the manual. The Array Tlateriel Command (AMC) and Training and Doctrine 
Command (TRABOC) have recently joined forces to conduct a demonstration 
project using this approach for concurrent development of improved techni- 
cal manuals and training programs for the tank turret mechanic. Smith 
describes the major steps in- this demonstration project as follows: 

Step I Job Analysis - This step is taken to insure a full 
and accurate definition of the job, as it is actually performed 
in the unit. 

Step II Task Analysis This step will identify the specific 
activities to be performed, such as the detection, removal, or- 
replacement of a faulty part on the tank turret. In this analysis 
optimum procedures will be developed by skilled technicians. It 
Is tnrough these optimum procedures that the need for time consuming 
decisions will be eliminated from the mechanic's job, ' 

Step III Structure Documentation and Training - At this point 
(Step III) a strategy for training and documentation will be 
developed,' Given t be tasks and optimum procedures developed in 
Step II, Tasks are identified for training site and media selection. 
This step will identify tasks to be included in the training 
program, or. in tho manual. 

Step IV Develop Dndum ent ation and Training - The methods for 
conveying the maintenance information are developed. The actual 
lesson materials and technical manuals will be fleshed out. 

In this last step, perhaps the most important of all, will te 
validaLion. It is here that the training and technical data are 
soldier tested to insure ability to perform the job. 



The demonstration will serve as a learning vehicle for the 
Army, A full evaluation will be made to document: the benefits 
and lessons learned so that applica-tion to other systems can be 
made, (Smith, H, L,, 1975, pp. 104-1Q5.) 

Smith does not elaborate on how the nex^ manuals are to be user- 
tested; that is, whether, as in TEC, the soldier testing will be part 
of the, developmental cycle, or, as in the current Air Force studies 
(Klesch, 1975), the soldier-testing represents ^ summative evaluation 
of the relative cost and performance effectiveness obtained under 
different manual and skill level conditions. 

Past Army Research on Manual Effectiveness 

. Several past Army research and development efforts have produced 
new manuals as a component of experimental training programs (e,g,, 
Gebhard, 1970; MacGaslin, Woodruff & Baker, 1959), In some instances 
the manuals were developed to serve both as a training aid during 
training and as a performance aid once on the job. Criteria used in 
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thase evaluations have been performance on selected job tasks while 
using the'saine job aid or manual (e:Kperii]iantal or conventional) the 
soldier used during /job training (experimental or conventional). 
However, since the^e manuals were developed as a component of the over- 
all training package, the evaluations conducted have focused on the 
effectiveness of the overall training package and not on the manuals 
per se, ' 

The one reported Army research effort, of which this OTiter is 
aware, that compared effectiveness of a new manual against an existing, 
conveufioaal manual .-Iso used ,perf omzance on several selected job tasks 
as the criteria (Rogers & Thorne, 1965). In this study, new graduates 
of a missile mainte^iance course were carefully paired and effectiveness 
of an experimental troubleshooting manual was studied by comparing the 
performance of tho.se using the manual with the performance of those using 
the conventional manual and notes used during their training. 

Performance testing of ^ the type referred to above is expensive. It 
is considered necessary to R&D projects that'are seeking to establish the 
effectiveness of new techniques or approaches. However, it is too cumber- 
some and expensive to incorporate into an operational literature quality 
control program. In addition, evaluation studies using performance tests 
as^a criterion usually cannot be conducted at the individual's work site 
ancl^generally require thQ.t he be brought to a special testing area and 
perform under various other special conditions. As a result, in thiq 
type of study we learn very little about the operational utility (or 
useability) which the manual possesses for the man on the job. 

Suggestions for Future R2search on Useability of Printed Materials 

^We need to develop relatively inexpensive ways of assessing the 
useability of printed materials and of identifying design features that 
maice significant differences in useability for given user groups. But, 
before we can make progress in this regard we need to identify what we' 
mean by the term ^'useability'^ and the conditions under which it can be 
assessed. For example, perhaps the term "useability" can best be - 
considered as a global term used to subsume the following characteristics 
^of the printed material when evaluated in relation to a given user \. 
located in his expected x>7ork or training site: *' • 

1. Design of rarmual for convenient work-site storage. 

As Highlander (1975) points out, size of a manual should be 
chosen based on its purpose and how it \7±11 he used. While this seems 
ohviouy, it. is a consideration that is apparently frequently overlooked. 
Manual designers and developers are not at the work site, "According to 
Highlander, manuals for Individual weapons which provide no reasonable 
storage space should probably be pocket-sized. However, a manual for a 
combat vehicle, which is required to carry a logbook, should take 
advantage of the largc:r logbook size since it can be'stored with the 
logtook and always be with the vehicle. , . 
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2. User's knowledge of, or training in, how to use the manual. 

People generally learn how to read the different type of infor- 
mation displays they coinmcnly encounter. Initial performance with a new 
type of information display, e»g., the first encounter with a logic tree, 
wxll. likely produce confusion and, if useage is on a voluntary basis, may 
result in avoidance of the manual. Studies of various new types of job 
aids have either incorporated them into the training program, or, as in ' 
the case of the Air Force (Klesch, 1975), provided short periods of 
training on how to use the job aid. Army researchers have also recognized 
this need (Shriver & Trexler, 1966) • 

3. Ease of locating specific procedures or information in the manual. 
This is another user problem area cited by Highlander (1975). 

VThile part of the problem may be 'lack of knowledge\ind practice 'in using 
maexes. Figures 6-A and 6-B illustrate how this can be made a tedious 
task or a relatively simple task. 

4. Ease of reading or comprehending specific procedures or passac^es 
of information. • , r- ,o 

The problem of reading difficulty level have been discussed 
earlier in this pr.poi . With" the shift to performance-oriented printed 
materials, however, narrative text has been sharply reduced in favor of 
what are, frequently, annotated illustrations. Figures 7-A and 7-B 
^^^^"^^^^'^^"^ ^.^-^^ contrast in design of the information display, \ghat 
effect dpes this change have on the user's comprehension of. the infor- 
mation ^presented? We can predict comprehension of the printed narrative 
m figure /-A by using readability fornalas. How can we assess and • 
predict^.^ comprehension of the information presented in Figure 7-B? 

5. Eai^e of L ransf orming information or procedures to the form 
required for task performance. 

Improving ease of transforming information or procedures to the 
form required for task performance is the core objective of performance- 
oriented literature. Detailed analyses of job tasks are carried out to 
provide the basis for preparing this literature. Can the extent of 
achievement of this objective be assessed from the perspective of the 
user? 

6. Completeness of task-relevant Information., 

This ^characteristic splits out in two ways. One way concerns 
inadequate information given in a procedure. For example, compare the 
information presented in Figure 7-A with the corresponding content in 
Figure 7-B. If you are a totally inexperienced tank gunner but capable 
reader, which would you find most helpful? The second way concerns 
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THE TANK GUNNER'S GUIDE 
(Tank, 90-mm Gud, M48A1) 
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Figure 6-A, A Table of Contents - "Before". 
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Figure 6-B. The Same Table of Contents - "After". 
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information scattered throughout a given manual ox across several manuals 
This issue has been addressed by recent attempts, for exaiaple, to collect 
all crew member operator information into a single manual for a given 
crew-served weapon (Braddock, 1975) • 

• 7* Accuracy of information. 

This is obviously an important characteristic and one that is 
dealt with extensively in papers presented at a recent symposium on 
technical data sponsored by the National Security Industrial Association, 
Validation for accuracy is expected to be accomplished during the equip- 
ment development cycle. However, it is clear that achievement of this 
objective is much more probable after the equipment has been in service 
for some period of time. 

8, Ease of updating manual. 

This characteristic is one that achieved frequent mention in 
the user survey . conducted by Showel and Brennan (1976) , 

9. Readership or extent of useage of the manual. 

Manuals may be provided for every job in the Army. However, use 
of these manuals by incumbents of different jobs would be expected tjo 
vary as a function of the nature of the incumbent's job tasks, the 
incumbent *s supervisor or other external sources such as Department of 
the Army requirements for passing Skill Qualification Tests. In the 
midst of this static, however, one would expect* higher readership to 
be associated with the more "useable" manuals. 

If we can make progress in conceptualizing "useab-? lity" and * 
ideudlfying design features that make significant differences in 
useability, then the next major task is to develop the capability to 
predict useability from design- plans for manuals proposed for use with 
a given user group. 
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COMMENTS ON THE PAPER BY K.rRN 



.Michael Macdonald-Ross 
The Open University 

Macdonald-Ross made many of his comments while critiqueing figures 
Kern h.^.d presented from a guidebook for Army writers. He made several 
points regarding design features of texts: headings, white space, and 
typography are supposed to be used as signposts telling the eye what is 
important to look at, and facilitating the eye's examination of the page; 
headings are supposed to tell the mind what the coatent is. Improper 
use of these design features leads to confusion. He -ointed out that 
numbers mean order, though they are frequently used sii^iply to itemize or 
to separate ideas. The latter uses can lead to a proliferation of number 
and again lead to confusion in reading. A second. major point made was 
that expert knox^-how exists in the form of design 'specialists , expert 
editors, typographers, table setters, and illustr^^ tors .. Much of the 
knowledge these expert craf tspersons have implicit or tacit, and he 
suggested that researchers should study these people and articulate 
their know-how in an explicit manner so that bet ter . guidebooks and 
training programs for text .producers .might be de-v^ioped. 
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TECHNICAL ORDER USEABILITY 



Robert C. Johnson 
Air Force Human Resources Laboratory 

The useability factor has not always' been a prime consideration in\ 
the development of technical data, Until recent years, the basic require-* 
ment^ f or, technical data war to describe the system or component. The 
maintenance requirements w^re identified, and t.o varying extents some 
detailed information was gi\*en on h?^-^ to actually accomplish the job. 
The traditional Air Force Technical Order (TO) is a large, bulky, hard^ 
bound book full of words and some pictures. It does contain an enormous 
amount of information. Much of the time, however, the necessary data is 
not easily retrievable. All too often the data is not easily understood, 
or complete, when it is located. Specific instructions on how to do a 
job are of ten virtually lost In the s.ui^rounding paragraphs of system ' - 
theory and description. Because of their large size and the bulky binder 
they are placed in, most TOs will not stay flat and open at the vork site, 
and are generally too big and heavy to encourage much use on the flight 
line. 

In the past several years, the Air Force Human Resources Laboratory's 
Advanced System Division (AFITRL/AS) and many other organisations have been 
X)7orking to improve the content and useability of technical data. One of 
the most popular new data concepts is generally referred to as Job Perfor- 
mance Aids (JPAs) . , 

At AFHRL our work on useability has been part of a larger effort to 
develop JPAs. JPAs are a very special kind of TO. Having been designed 
and developed specifically to increase effectiveness and useability, JPAs 
have several very distinct characteristics. The most obvious character- 
istics are the small, pocket size, the step-by-step detailed procedures, 
and the extensive use of good illustrations (See Figure 1). AFHRL work 
in JPA research over the past 12 years has led to other developments that 
are not as obvious to the user, however. Many of these less obvious 
developments actually provide the real strength of JPAs, 

HOW TO DEVELOP USEABLE DATA 

The following processes'^ are required by our draft specification 
for JPAs, contained in AFHRL-TR-73-43. We believe that they are essential 
CO the development of complete, accurate, and useable JPAs, Many of these 
processes would also be applicable to the development of traditional TOs. 



"Specific details on the content and utilization of these processes are 
contained in AFHRL-TR--73-43 (Joyce, et al,, 1973a). The AD numbers are 
iVD7757G2 and AD7757065 or they may be obtained from: Mr. Robert C. 
Johnson, AFHRL/ASR, Wright-Patterson AFB, OH 45433, Autovon phone 
785-2606, 
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T,0. 1C-14:a-2-40G-4 ; " I 
REMOVE AND INSTALL FUEL PUMP I 
Install Fuel Pump On Engine: | 

CAUTION i 

I 

Do not use any lubricant, I 
other than engine oil on' j 
mating splines {!). Use of i 
grease can olog oil feed J 
holes in engine gearbox and'; 
restrict oil flow during 
engine operation. ; 

6. Lubricate fuel pump mating ' 
splines (l) with engine oil 
MIL-L-780B. 

CAUTION 



Do not allow weight of fuel 
pump to rest on shaft 
splines (1) during 
installation. " Undue weight 
can impose stress on gearbox 
shaft oil seal, causing seal 
damage . 



7." Working with assistant, ■ 
raise fuel pump into position. J 
Align pinholes (4) in pump with; 
pins (3) on gearbox mounting / 
flange (2) . ' j 

3. Install fuel pump on 
gearbox mounting flange (2) . 



T,0. :c-I41A-2-4JG-4 




Figure 1. Example of Job Performance Aid Format 
and Use of Illustration. 
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A maintenance task analysis - This process forces a thorough, 
accurate, systematic look at the tasks required. Several intemediaf.e 
products are required to iri^ure that all tasks are identified and 
analyzed • 

2* A formal user assessment - This process identifies and describes 
the expected user of the TO. The object is to gather as much information 
about the typical user as is reasor.ably possible in order to write the TO 
to fulfill his specific needs* E^q^erience, trainings aptitudes, reading 
levels, etc., are examples of the kind of data coll ected, 

3- A JPA/ training tradeoff - This process involves the JPA contractor 
the Air Force procuring agency, and the Air Training Command- The object 
is to identify what information will be taught in training, what vill be 
contained in the JPA., and what should be covered in both training and the 
JPA. It also requires close cooperation between the contractor's technical 
data and training divisions. We feel that this helps ensure complete data 
coverage in either training or JPA, and results in more realistic, accurate 
data for both. 

^* The level~of~detail guide - This process is a set of guidelines 
for deciding how„rauch detail is required • These rules are developed in 
consideration of the task analyst's knowledge of the system, from infor- 
mation from the user assessment and the JPA/ training tradeoff processes, 
and from the desired quality of . performance • 

5» Writing requirements - In addition to. the level-of-det:ail guide 
previously described, specific attention is. giv,'en to how procedures are 
written. We ask that the writer always consider' the user. We recommend 
that the steps be limited in size to no more than three sentences and no 
more than thirty words. No more than fifteen words are permitted' in any 
one sentence. These limitations are guidelines, rather than absolute 
requirements. It is desirable to observe these limitations whenever 
possible. A standardized verb list ensures that the same verb is always 
used to describe a give desired action. It is also highly recommended that 
the writer have access to the hardware so he can check out each procedure. 

6- Use of illustrations - A major factor in the success of JPAs is 
the extensive use of good illustrations. We recommend that an Illustration 
be on the same page as the procedure, or on the facing page. Each time a 
part or a location is mentioned in the procedure the reference number must 
be included to enable the user to locate the reference on the illustration. 

The illustration should be a line drawing, and it should fully support the 
procedure by identifying each part mentioned in the procedure. When 
required, it should locate the illustrated components on the next higher 
assembly. Care should be exercised to prevent overcrowding the illiustra- 
tion, and to avoid presenting excessive detail. 

• Quality assurance - Specific quality assurance provisions are 
required to ensure the required quality level . Ir.cluded are reviews 
during the development process and actual cryout of data first by contrac- 
tors, then by the Air Force. 
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8. Intended use - The x^iter should detemine specific use require- 
ments for the manual. Items of interest include where it is. tc be used, 
how it is to be used, what activities it is expected to support, and any 
special demands likely to be made on the manual. 

C-141 JOB GUIDE TECHNICAL ORDERS 

The first large-scale use of the JPA concept in an operational 
environment is that of the job guides developed for the C-141 aircraft by 
Westinghouse Corporation. These job guides were developed according to 
MIL-M-38800A, The using command^ Military Airlift Command (MAC), and 
the contractor, Westinghouse Corporation, both agreed that a task analysis 
was required. These TOs, then, represent a compromise between the rather 
relaxed MIL^M^38800A specification and the more specific, controlled 
development procedures favored by APHRL and described in APHRL-TR- 73-43 • 

The C--141 guides cover on-equipment work only, and incorporate logic 
trees for flight line troubleshooting. These TOs are in the .4 x 8" size 
for everything except the troubleshooting logic trees, which are con- 
tained in the traditional 8-1/4 x 10-3/4'' size books. 

These TOs are currently being service-tested at Charleston and Norton 
AFBs. This 90-day test is preliminary to Military' Air Coinmand (KAC)-wide 
introduction in January 1976. 

C-141 JOB GUIDE EVALUATION 

At the request of HQ USAF, a team' has been formed to evaluate the 
C-141 job guides during the service test period, and for the first 10 
months of their use M\C-wide. This team consists of representatives from 
MAC, from the procuring agency, Air Force Logistics Comrnand (AFLC) , from 
the Air Training Coiranand (ATC) , and from the Air Force Human Resources 
Laboratory (AFHRL). AFHRL responsibilities in the evaluation include 
user acceptance and training. Our study of liser acceptance includes both 
attitudes and useability. Both will be described in the following 
paragraphs . . 

. Attitudes - Our primary interest is to determine if any major shift 
in attitudes occurs as a result of greater experience with the job guides. 
To establish a base line we collected data at Charleston and Norton AFBs 
oeiJo/Le'job guides were implemented. We used a questionnaire that 
explained and illustrated a job guide^ .than asked questions to determine 
their attitude toward proceduralized data. Our sample included 124 main-' 
tenance personnel at each base. .The sample was limited to the job skills 
of personnel xmo would be using job guides, but was not controlled by 
skill level, rank, time in service, or shift worjced. Every effort was 
made to get as representative a sample as was possible Xsrithout unduly 
interrupting the maintenance activity. We will again gather attitude 
data at both bases in late November or early December when the service 
test will end. In addition, some attitude data v;as. collected by 
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interview during the initial implementatioa period at each base- We 
want to determine if there is a major change in attitude after repeated 
exposure to job guides. These data should be analyzed and available for 
discussion early in 1976. 

Useability - Our goal in the iiseability study is to determine what 
positive and negative aspects of the job guides surface as a result of 
their daily use in an operational environment, 

USEABILITY CHARACTERISTICS 

In its most basic concept, we think of useability as the factors or 
characteristics of technical data which encourage their use by the 
intended user in the intended environments We feel that to be useable, 
a TO must have the following characteristics: 

.1. It must be technically correct. 

2. It must be easily understood by the user. 

3. It must include all necessary data. 

4. The required data must be easy to find, 

5. The data must meet the needs of the user. 

6. It must be easy to carry and use. 

7. It must be readily available, 

Each of these top'^^s will be discussed and the relationships x^ith 
the previously described development processes will be explained. The 
procedures we use to collect the data during the service test of the job 
guides are also described. 

Technically Correct - TO information must be correct. Today's Air 
Force requires that TOs be used on the job. The TO is assumed to be 
correct. Proceduralized, step-by-step data requires virtually 100% 
accuracy, because any errors become very noticeable in the detailed 
procedures. Because the user does, or is expected to, follow every 
step, any errors quickly erode his faith in the validity of the manuals. 
Faith in the validity of the TO is necessary if the TO' is to be actually 
used ori the job. We specify the use of a task analysis, strong quality 
control;- in-process reviews, access to the hardware, and 100% hands-on 
validation and verification to ensure the technical accuracy of JPAs, 
In the C-i41 Job Guide Evaluation, AFHRL is not involved in evaluating 
the technical accuracy of the data. MAC and AFLC are responsible for 
this aspect. We do- of cour^ note and report all technical discrepancies 
that we discover. ) 

Easily Understood - Temnical data that cannot be understood is of 
little value to the user. ^Readability levels are quite important here. 
We have, however, "taken a slightly different approach, and also enjphasize 
the value of the illustration. We require a very close relat^op^ip 
between thfe proceduralized- step and the accompanying illustration on the 
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same or facing page. Each time, a part or location is mentioned in the 
procedure, it must be identified on the illustration. The callout number 
is enclosed in parenthesis in' the procedure to identify and help locate 
the part or location* The first time a part or location is mentioned in 
the procedure, a location illustration is provided to pinpoint the'loca- 
tion on the aircraft or on the next higher assembly. This hklps ensure 
that the user can locate that part or area. In "an attempt to increase 
the understanding of the user, we have developed a standardized verb list 
(See Table 1), Use of this list ensures the same verb will 'be used each 
time that particular action is desired, A preliminary information page 
is^.a part of every task (See Figure 2), This page contains all of the 
information the technician requires to start the job, including the 
special tools required, necessary manpower, replaceable parts and consum- 
able supplies, and a brief statement about the task. Because the level 
of detail of the procedures also affects the understanding of the user, 
we suggest taking a dual-level approach* In this approach, very detailed 
step-by-step data is provided for those who need it, and the outline or 
checklist level of data is highlighted for use by the more experienced 
technician. 

We are evaluating this aspect of useability. By observing the job in 
progress, we identify areas of confusion or misunderstanding. We also 
interview the technician immediately after the job is completed. Areas 
of misunderstanding are identified, and the reasons the confusion occurred 
are determined. 

All Data Includec - AU of the data .required to do the job should be 
included, in the procedur-^^. As much as possible, the book should be 
complete within itself. Torqur. values, instrument readings, pressure 
and electrical values expected at any given point should be provided. 
Referencing to other manuals is discouraged, and reference within the TO 
should be held to a mininiuin. The intention is to make the TO complete 
e- gh that the user doesn't have to refer to any outside source of 
iiiiormation to complete the job. 

Observation of the job in progress permits us to identify instances 
v;hnjn the technician searches for information in other TOs or from^ther 
people. 

Easy to Find Data - The organization or arrangement of data within a 
TO ic: an important f-actor in useability. A study by Losee, Allen, Stroud, 
and Verjlulst, (1962) for the Aerospace Medical Research Laboratories found 
that 78% of the maintenance people questioned estimated that they spent 
more than 10% of the total job Lime seeking information in the TOs ifn 
order to work on an unfamiliar piece of equipment. We at.AFlIRL just 
recently replicated a portion of that study. The data cre*^still being 
analyzed, but a preliminary review indicates that the percentage of time 
spent in searching for information is probably no better now than it was 
thirteen years ago. The task of finding appropriate information in most 
TOs can be a frustrating one. There are several obvious reasons for this. 
(1) The information is presented or organized "in a pattern that may be 
logical to the witer, but r .t to the u<;er. The writer often thinks and 
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ftHHOVE AND INSTALL RSCEIVHR- 
TRAH5M»T7£R 

SECTION I 

PR£LIMJHA;?r JNFORMATIOP* 

Introduction 

TMs activity covers th» cctrpUta 
removal from and Installation on the 
AN/TPS-IA of the Htfcalv0r-.Tran31wJ.tter. 
It provld»»-acco5»-t» the Elevation 
Prlve, the ElevaMon Foilovor^ the 
Yoke Slip Ring Ass^rbly, and ttie 
Az'imjth Drive. Install R-T Is part 
of the site set-up procedure. Rfiirove 
R-T is part of tha site dlaassefl^jly. 
TTjo Ramov© activity Is aUaya pro- 
ceeded by Rerpove Antenna, Voliro 
p2S<t.5-l. The fns^^ll activity I3 
a i way J preceedwJ by Insta.M Yoke and 
ElovaHon Drive, Volcne 2-5, pyje 3-1. 

Applicable Serial Uurrb^rsz 

All 

Special Tools and Tss t .gquipA^nt : 
Suppl *es : 

Sealant, Loctite MlL-S-22473 
AFTO Tom 3*9 
AFTO From 350 

Personnel Rgqulrcd : 

One Technician, 30332 
Three Assistants 



ACTIYITT IN0&^ 



Procedure -^^ 

Rawove Receiver- ' 
Transmitter 

Install Receiver- 
Transmitter. 



1-2 

I-rlO 



R£PLACEME^^r PARTS 

9 0-rlngs, 5AC0I 

2 Consll Caskets, SAtGI, and C2 
4 Con.nector Gaskets, 5A^3^ 

3 Waveguide Caskets* 5A4G7>Gn 
2 Seals, 5A4SEI and SE2 



E- qulp^nt Conditions : 

/.'i^lvnna rctnov<fd, Volur^ /-6, page 5-1. 
AsltTvth Interlock eftjsi^jd. Elevation 
Interlock engaged at +90 d-gre«^. 
•^celver-Transmltter bottom cover 
reroved. 



l-l . 



igure 2. Example of Format for Preliminary Information Page. 
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writes in terms of a good working system, and successfully describes it. 
The user, however, nearly always approaches TOs from the view of a non- 
working system". (2) The actual format of the TO often hinders a data 
search. The block text style requires reading virtually every word of 
a paragraph identified by the index. (3). The writing style is often so 
cluttered with words that the clear, concise answer the user needs is 
hidden. Thus, the writing style coupled with the user's reading capability 
and impatience often hinders the data search. (4) The index is incomplete 
or inaccurate. This makes it difficult to even locate the approximate 
area without paging through the book. 

These areas deserve some attention from To' researchers. At AFHRL, we 
have tried to address these areas in our JPAs. We ask that a task 
analysis be done, and that the viewpoint of the ussr be considered through- 
out. The task analysis forces a thorough, logical approach to data 
development. ' In arranging the data into volumes, we require that the data 
be arranged by sub-system, and that, in general, the tasks be in probable 
or most likely sequence of occurrence. More work needs to be done in the 
organization or arrangement of information within a book. 

The format of JPAs makes it relatively easy to locate specific infor- 
mation. Because the data is broken doTO into short steps, the desired 
data stands out clearly after the index has referred the user to the 
appropriate section. 

The JPA requires the developer to limit the size of the steps. As a 
general guideline, no step should contain more than three sentences, and 
no more than thirty worfls. No sentence should contain more than fifteen 
words. In addition, to decrease misunderstanding a standardized verb 
list is used, as is a specially developed noun list* We feel that these 
steps help reduce the wordiness common to many TOs. 

The index is another area that deserves specific research. Our only 
effort here has been to use the task analysis data as a basis for the 
index, and thus make it more complete. We also require the common, every- 
•day names of an item be included in the index, along with the formal 
engineering name of the part. We feel that the index shoild be alphabet- 
ical by part, not by system or sub-system (See Figure 3). Special Emphasis 
is placed on the index during validation and verification. 

Observation and Interviews are used to evaluate this area of interest. 

We observe the technician as he first tries to locate the proper procedure 

in the TO. Interviews are used to determine how easy it is to find 

specific bits of information in the job guides. 

Match the Needs of the User - The needs of the user must be considered 
during the development « of the technical data. We approach this problem by 
requiring a task analysis that identifies every task to be accomplished at 
a given level of maintenance. A JPA/training trade-off identifies which 
data will be taught in training courses, which will be contained in the TO, 
and which will be covered in both training and the TO. A user assessment 
process provides data which describes the intended typical users of the 
data. This user assessment data provides valuable information to the 
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EgUIP^!S^iT AU=HAaeTICAL UiDSX * 

♦ ' * . 

Air Clgahe r, I3206H0250 

Service . i 

Afttenna, Radar, 70il-7!00-l 

Install on R/T 

Install In Transit Frame 

Rerrcve frcfn R/T 

Rsnova froo Trensit Frarr^ * 

A zintuth Drive , -7QI I -7500-1 

Assenbla 

C*i3ckout/Troub 1 ^hoof 
Dlsossemb Is 
R«3pa J r 

Battery ^ MS75b':7-2 

Char^o 
' 'C^pck eiectralyl-a 
Service 

Boro5Ccp6» Assefrhly , 701 1-7300-1 

Clean 
Insist)] 

■ Condenser Pressure Switch , 1463-94 

Adjust 
Conn^cforj , i93027J3 

Repair 

CrankcQsg Pressure Rgqulator y ERY 2600 
Adjust 

CrlffpH:n Contacts ^ P/N 91038-3, P/N 91042-1 
Repair - See Connectors, I93027J3 Repair 



Volume 



2-r2 



2-1 
2-5 
2-1 
2-5 



2-6 
2-32 
2-6 
2-6 



2-1 
2-1 

2-1 



2-6 
2-6 
2-6 



2-3 



2-70 



2-1 



3-f 



2-1 
-5-1 
2-1 
4-1 



I-l 

2^1 



3-1 



5-! 
4-1 
4-1 



3-1 



e-i 



7-1 



Figure 3. Format Sample for Equipment Alphabetical Listi 



ng. 
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JPA/ training trade--off and to the *level-of-detail guide. The level-of- 
detail guide attempts to provide the proper amount of detail for each 
task, according to the needs -cf the expected user. Some specific infor'^ 
mation provided by these procasses include vhat a typical user can be 
expected to know, and what he should not ba expected to know; what tools 
and test equipment he can use; the reading level, aptitude and intelli- 
gence scores for the t3rpical user; and general military and civilian 
experience applicable to the maintenance tasks. These data allow the 
many needs of the user to be met, whether in reading level, additional • 
detail to overcome lack of experience, specific illustrations, or tool 
and test equipment instructions. 

We use questionnaires and/or interviews to determine- whether the 

needs of the user have •been met. We interview the user immediately 

■. * ■ 

after he completes a jc^b. If possible, v/e'also have him complete a 
questionnaire. We try to do this before he returns to his shop area 
and gets involved in other tasks. 

Easy to Carry and Use - The physical characteristics of a manual 
are very important. Many of today's manuals are large, heavy, bulky 
books. Often the user must carry one or more manuals, his tool bag, and 
whatever supplies or te^t equipment he needs. Once on the job, the 
manual is too large to lay down in some areas close to the job, and in 
most cases, the manual v/on't stay open to the proper page. Illustrations 
and wiring diagrams are often printed on foldouts, which flap and blov, 
and eventually tear xn the wind. The paper on v7hich TOs are normally 
printed is not impervious to oil, water, and dirt, cind if used, soon 
becomes dirty and hard to read. Constant use causes the -pages to tear 
out of the binder. Lost or missing pages can invalidate procedures and 
thus increase the possibility of an accident or at least a Quality 
Control discrepancy, 

JPAs provide some answers to these problems. First, the 4- x 8" . 
size of the manual used on the. flight line is a very convenient size 
to carry and use. JPAs are also, bound in a binder that will lay flat 
and open, and it is small enough to stay with the user, even in tight 
quarters. Foldouts, while permitted, are discouraged in JPAs. We 
also eacourage the using command to demand the special paper that is 
impervious to oil, water, and dirt, and won't tear, especially for 
flight manuals. These things all encourage greater use by the mechanic. 
The smaller, more useable size is a well-liked feature of JPAs, 

We gather data on this subject primarily by observation- Our 
questionnaire also contains some questions on size, foldouts, etc. 

Ava ilab il i t y - A final requirement for a useable TO is that it be 
readily available when it is required for use. This factor is only 
partly xvithin the control of the developing agency. A smaller size of 
manual is easier to carry and handle and thus is more likely to be 
available when needed. Other aspects of availability involve storage 
location, numbers of manuals, useage policy, and transportation to and 
from the typical storage location. Some .organizations , such as the 
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Air Force Military Airlift Connnand (MAC), score the applicable maintenance 
TOs on board each aircraft; Organizations with fightar aircraft often 
carry the maintenance TOs ^in ^ line vehicle to make theiu readily avail- 
able. Other organizations maintain the TOs in each work center and 
expect the user to carry them back and forth to the job location. 

This data cair.e from observing where TOs are kept and how they are 
obtained. Our questionnaire also contains some questions about availa- 
bility of TOs. 

SUMMARY 

Useability ±s' a very important aspect of technical data, but good 
?iseability characteristics will not occur by chance. The development 
processes described in this paper and more thoroughly discussed in 
AFHRL-TR.-73-43 help ensure that the needs of the user are identified 
and met. 

The evaluation criteria presented here are being used at Charleston 
anrl Norton AFBs. Thus far they have proven to be realistic expectations 
for proceduralir.ed daca evaluations. It is anticipated that further 
refinement of both the development processes and the useability character- 
istics will occur as a result of this initial evaluation of proceduralized 
c-.ata in an operational environment. 

We believe that the results of the attitude and useability studies 
will ^:reatly influence future procurements of proceduralized data. 
Continued research of the development and utilization of proceduralized 
data will also benefit from these studies. 
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COMMENTS ON THE PAPER BY JOHNSON 



^Irnst Rothkopf 
Bell Telephone Laboratories 



In his i'coiEinents on Johnson's paper on the useability of documents, 
Rothkopf nain'point was that there is a need to reconcile the archival 
functions of technical publications and the job support functions of 
such publications within the same training system. Archival documents 
are frequently held to be source documents for all sorts of doctrine, 
standard operating procedures, and the governance of work. They are 
thus a sort of stable reference for many users. The job document, on the 
other h^.nd, is committ^ecV^'to a particular function, and that function may 
•change: job famili^e's may-be reorganized, systems may change, and so 
forthi. There is thus a n^ed for constant updating. Rothkopf saw this 
constant upidating as a particular problem for job aids. He raised the 
questions of whether job performance aids ought to be produced centrally 
and, if so, xv'hether updatin^s should also be produced centrally, or 
might local units produce their own updatings; he concluded that these 
are questions for future research on the useability of documents. 
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USEABILITY RESEARCH IN THE NAVY 



William G. Muller 
Naval Air Technical Services Facility 

The Navy today is actively involved in documentation research and 
improvement efforts to positively affect the useability of technical 
manuals. This paper will, however, discuss only four of the many "efforts 
presently ongoing within the Navy. 

As background, an attempt will be made to provide an appreciation of 
the scope of a technical manual program within one of the five Systems 
Commands, i.e., the Naval Air Systems' Command (NAVAIR) . NAVAIR was 
^^°.?_§Il.J?,^ cause it is the most complete, and it is the program with which 
the author has most familiarity. NAVAIR presently supplies technical 
manuals for 135 aircraft models. The active * inventory contains 25,000 
manuals. Tlie total page count approximates tlUit^ nii^i^OH poge^S, and each 
new system requires more and more pages. Figure 1 shows this growth over 
the past 30 years. Approximately 60 million 'dollars is expended every 
year to procure and update NAVAIR technical manuals. 

Technical manuals are not- written in-house. The bulk of major 
system publications are produced by the prime aircraft contractor. Such 
a contractor employs between 300 and 500 technical writers. In contrast 
to that, some manufactmrers who supply support equipment have a technical 
writing staff of one and that one may be performing other tasks as well. 
In 1975 NAVAIR had 2,200 contractors, large and small, producing technical 
Pianuals . 

The process of writing a technical manual is a long one. The time 
interval from the first step in the process until the actual publications 
are delivered is typically 3-4 years. Unfortunately, however, the config- 
uration of the hardware system is constantly changing and, as a result, 
the documentation system is in a state ot flux. As an example, on a 
recent aircraft procurement, the documenfe'ation process was started in 1969; 
preliminary manuals were delivered. in 1972; and final formal manuals were 
delivered through 1975, To date almost 1,000 engineering, changes have 
Deen approved^ All of these changes in one x^;ay or another impact upon 
the documentation system. To compound the situation even more, there was 
a significant change in the media for presentation of documentation. A 
decision v/as made in or around 1969 to convert .^rom paper to microfilm- 
This change was significant, and use is radically af fected Finally , 
there are those changes that appear and no. one is quite sure how to deal 
with them. One such c'-^ange is the presence of the all-volunteer service. 
With it came the potential for a lower reading ability. However, there 
was a change in the employment marketplace which interacted with the 
predicted reading levels. All of these changes create concern for those 
trying to build useable technical documentation. 

The Naval Air System Command in 1970 realized a need for a research 
and development program in technical .documentation. An analysis was 
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Figure 1. Technical Manual Growth Rate Within 
the Naval Air Systems Command, 
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undertaken to determine just what 'area in the documentation system would 
offer the greatest payoff. Another way of stating that is — what are the. 
essential elements in our system and where would available . resources best 
be utilized? 

Figure- 2 is a simplistic view of the documentation system. Three 
•key elements are shown: the technical writer, the technical manuals, and 
the reader/user. Outside of the technical manual area, the flow is from 
design to manufacturing to the delivery of the prime product, i.e., the " 
hardware. A secondary output of design a^d manufacturing is a data base 
which serves two functions. It is- first an engineering tool for design 
and later for modification and secondarily it is the starting point of 
the documentation effort. The data base is the initial input to the 
technical writer, who converts the data into maintenance information 
presented in the form of a technical manual. The technical manual is an 
aid to be used by the technician who is initially a reader and later a 
user, all in the support and maintenance of the hardware. But what is 
the locus of our technical manual problems? Each of these major compon- 
ents was examined, starting -with the user and working backward. 

The reader /user establishes technical manual requirements. He or 
she essentially sets a significant portion of the Xvrarrant for our docu- 
mentation system. Reading level, comprehension Jikill, and anticipated, 
training proficiency are all considerations. In addition, the work 
environment and the type of equipment or hardware are significant factors. 
Attrition is also a problem, but it is something that little can be done 
.about directly. All of these factors must be kept in a very delicate 
balance. If . they are not, the credibility of the documentation will 
suffer and when that occurs, the iQanual will not be used, or it will be 
used only as a last resort. Unfortunately xjord of mouth, recall from 
training, and trial-and-error experience can become the primary mode of 
coniiuunication when a technical manual lacks credibility. As a result, 
its useability can approach zero. 

The technical manual as a vehicle for communication is essentially 
concerned with the content and format of the delivered product. Uhat 
information should be in a technical manual, i.e., its content, is 
simple: all and. only nhat information required to perform the task. 
The decision on what information to include is made by the technical 
writer, who will be addressed shortly. A process called validation and 
verification is employed to^ insure that the content is accurate, but 
this occurs a^teA the manual is XN^ritten.- In terms of the format of the 
technical manual, the available approaches are innumerable. There are 
literally hundreds of available formats. Unfortunately, there is a 
propensity on the part of requiring activities to select a single format 
and to apply it universally. The reason for this is understandable as 
there do not exist today the tools or techniques to select and differ- 
entially apply formats. Taere has been an enormuus amount of research 
and evaluation into format. ' Typically, a new format is tested against a 

conventional" approach and the dif f erences ,\ when not masked by experi- 
mental error, are mixed. The essential quality , or that specific aspect 
of the format that could lead to a prescription for the format, is 
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seldom teased out of the data. As a result there has often been a go or 
no-go decision to apply a particular format across the board. Some 
recent efforts are being undertaken to alleviate this all-or-none 
approach. They will be presented later in'this paper. An additional 
ractor that must be addressed, is readability requirements. NAVAIR 
advocates the use of graphics in either a ' complementary or redundant- 
method to facilitate the reading and comprehension of the material. 
Microfilm, when used, dictates "work packaging" which presents all the 
material required within the confines of a lijnited number of pages or 
rrames and with minimum referencing. 

_ _ The last element in documentation production is the technical writer. 
It IS here that the single largest contribution to useability can be 
rea.„i2ed. To this point, we have been dealing with the reciDient of 
information and the vehicle of communicating the information. NAVAIR ■ 
believes. and operates ' under the principle that in communicating infor- 
mation, the responsibility lies with the sender. The sender in this 
system IS the technical writer. The life of a technical writer is a ' 
veritable nightmare. The technical writer is required to convert an 
everchangmg data base into a technical manual in a single formal. 
Direction is provided through a legal document known as a specification, 
which IS typically embellished by a company style guide. A potontial 
solution presently under denelopraent in a single writing guide, '^ha ' 
technical writer is ofcen just one of a hundred workers and, thereby 
sees only part of the total documentation. Ite has varying impressions, 
and I emphasize ^np/Lg^i.^o^l^ , the reader/user and the working environ-, 
ment. oub.iectivity must be removed from the process and supplanted by 
objective methodology and techniques. The ability to verify .siting 
against actual hardware is nonexistent for the greater part of the effort 
In-process measures of performance, must be available to the writer. ' 

That is in summary some background 'on documentation system problems 
It IS irom this viewpoint that the requirements for the research and 
development efforts in useability are derived. It is- obvious that what 
IS needed are techniques and tools for the technical writer that will 
permit the identification of . the data available and to be acquired, and 
that will permit selection of the proper format based on an explicit and 
objective cnaracterization of the user and the conditions that affect 
performance. Four ongoing efforts to address these requirements will be 
■described m the remainder of th.Ls paper. 

Navy Technical Manual Sy.stem Program^ 

The Navy Technical Manual System (NTMS) , now in exploratory development 
with planned completion in the 1980 time period, is a program for the develop- 
ment of a system of integrated procedures and equipment to improve the util- 
ity, management, production, distribution, storage, and update of operator and 



"^The Navy Technical Manual System (NTMS) was retitled the Navy Technical 
Information Presentation Program (NTIPP) In May 1976. 
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maintenance technical data (see Sulit, 1975). The NTMS program will (1) 
quantitatively evaluate the capabilities and deficiencies of technical 
manuals (TMs) in the Navy, (2^ evaluate the current and future needs for 
N^vy TMs, and then (3) design, develop, and test an integrated Navy 
Technical Manual System which will provide effective and timely operator, 
maintenance, and training data* NTMS will also provide improved operator 
and maintenance performance aids for the fleet, training community, and 
shore establishment. 

NTMS is in its second year and is presently in the process of 
selecting a contractor for Phase 1, which is the concept formulation 
stage of the project. The NTMS has as its goal the integration of the 
various elements -of the Systems Commands' normal operating systems. 
This means that common functions will be , standardised unless they are 
performed differently for a specific reason. In addition, changes in 
the state of the art that can yield a better deliverable product will be 
incorporated. NTMS has developed a working model (see Figure 3) contain- 
ing the essential elements of a technical manual program. These elements 
will be thoroiighly examined and certainly expanded as the procr:im develops. 
Because the evaluation of the proposals for contractor selection is still 
underway, further elaboration will not be undertaken in the paper. 

Writers Guide 

Technological advancements in aeronautical systGim desif^n have trans- 
formed the casually regarded technical nanual into a desirable maintenance 
tool. In many cases, its use has become mandatory. In th^ise instances, 
the maintenance technician can no' longer make decisions on specif i^c main- 
tenance actions x^;ithout relying on the technical mrinual. Althou(-;;h routine 
maintenance actions will continue to. be performed without reference to the 
written word, new support concepts , automation, and micromlniaturizatiou 
of system equipment have made the technician heavily dependent on the 
manual during the performance^ nf his duties. Because of this stciidy 
increase in the use of the technical manual, greater emphasis is being 
placed on data credibility and overall useabiliry of docuraentation. 

The quality of the documentation delivered to the fleet has always 
been of interest to those responsible for its preparation. However, thq 
renewed interest in technical manual^development has generated writing 
and readability studies that have shown thar: the existing manuals are 
less than interesting to read and are written at a reading level that 
tends, to impair comprehension. Further, the volume of data in conventional- 
style manuals complicates data arrangement, making the desired information 
difficult to locate". It has become increasingly evident that simplified 
presentation techniques must be .developed to enhance useage. 

Technical writers, editors,' and illustrators will require, special 
training to assist them in understanding and complying; v;ith these uex'j ■ 
and innovative techniques and requirements. The development of a writer's 
guide was undertaken v;ith the intent of assistin^i in the es t.-i]>lishment of 
standard practices that wouli ensure preparation of more x-titr- f^..s ting 
documents and also meet accepted useability criteria. It i.G intended that 
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the guide be used in conjunction with the existing technical manual 
specifications. However, the initial issue was specifically prepared 
•to satisfy the urgent requirements of the task-oriented "Work Package" 
technical manual, • 

Military technical manual preparation specifications provide con- 
tract definition of the two major data requirements: format and content. 
These specifications generally define the basic format and content of the 
completed technical ip/^nual. Statements such as "the prepared document 
shall follow the ma nance concept and support policies" serve as 
guidance; detailed ' • to" information is not provided. The specification 
becomes a vehicle fo^ ;pnr.tion and acceptance control by quality control 
personnel, rather th n ^ detailed development guide for the preparing 
activity. This t:.::^lure to provide detailed preparation policies and 
procedures has ,v :i: : zechnicax writer extensive latitude in writing 
style, and the - . or follows only vague guidelines as to what con- 

stitutes a well-i. I ..cd illustration. As a result, the editor has been 
left with the problem of interpreting subjective decisions made, by the 
v^riter or illustrator relating to their interpretation of the various 
specifications* 

Many contractors have attempted to fill this void with company- 
prepared styl^. guides . The major thrust of these guides has been specifi- 
cation interpretation and sl:::nd ardization. X-Zhether intentional or not, 
the company guides tend to meet minimum requirements that ftilflll contrac- 
tual obligations, rather than providing innovative improvements in docu- 
mentation presentation., A major drawback in the use of any specification 
is that while it directs the quality of the product, it also restricts 
incorporation of improvements that were not anticipated when the specifi- 
cation was prepared* Because of variations in interpretation aad differences 
of opinion, major conflicts are evident when company style guides are 
reviewed and compared. In many caseSj these * apparent conflicts betWeen 
company style guides and governing documents are the result of the 
contractor's attempt to respond to special requests by the government for 
specific types of unique, publications; the style guides interpret the 
official guidance in relation to the specific products required. 

In an effort to improve this situation, the Naval Air Systems Command 
has developed a writing guide (Naval Air Systems Command, 1976) to provide 
a single' standard for preparation techniques as well as a supplenien.t to 
the various specifications in force. It is intended to assist the writer, . 
illustrator, and editor in the development of the best possible technical 
manual product. Task analysis, content outlines, and data arrangement y 
are defined. Selection of logical options is cixplained, as are wnriting 
and illustrating techniques* Perhaps the major innovation is the incorp- 
oration of criteria affecting readability and comprehensibility require- 
ments. The information contained has been prepared xWith the intent of 
guiding, rather than restricting, the writer *s freedom of expression. 
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This guide was prepared to facilitate the use of separate sections 
by interested readers. The coverage includes these types: 

- the objectives of the Work Package Concept; 

- the development of a technical manual , regardless of format used; 

- product analysis, task identification, and task analysis related 
to the planning of t. technical manual; 

- the davelopment of new writing techniques; 

the preparation of comprehensive illustrations with emphasis on 
microform compatibility; 

~ techniqua^ that w* 11 ensure credible and useable documentation, 
through the use of compreheusibility and readability checks. 

■ Tlie Work Package (\>rP) Manual' is user-oriented. It improves on the 
conventxonal manual in three ganeral areas: organization, format, and 
comprehensibility. 

WP manuals are organized by functional task. The optimum will 
provide all the information reruired to perform a specific maintenance 
action in- one package. This will not always be possible. However, if 
the information cannct be provided in one it will be provided in a 

series of Ws. The primary task and all associated tasks that mav be 
required to complete the job are organized in a manner that will rfiiquire 
a minimum of searching by the technician for his information. I:i a 
conventional manual, which is organized by type of information rather 
than by task, it is often necessary to search through several manuals, 
volumes, or sections to locate the needed information. 

The conventional manual, x^jlth its varied format, and small type size, 
has provided significant problems for the microform (microfilm or micro- 
fiche) user. The format of the WP manual is designed to ensure total 
compatibility with either production media: paper or microfilm. Minimum 
type sizes have been stipulated for control of legibility. Illustrations 
must be properly programmed to support text. - Wiring diagrams and 
schematics are tightly controlled. Line drawings are used in lieu of 
photographs. 

Tlie use of this guide Xsrith W preparation specifications will add 
comprehensibility assurance criteria specifically developed for the 
improvement of documentation usi-iability . The goal is to simplify the 
presentation of technical material and make it easier for the reader to 
understand. Thi? simplistic approach has been developed . through the 
application of rules covering siting style, use of preferred words, and 
comprehensive structuring of text material strongly supported by comple- 
mentary figures and tables. Although developed for WP manuals, the 
comprehensibility assurance criteria can be applied equally well to all 
technical manuals. ' . ' 
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Figure h is a sutmary of the couiprehens ibility assurance criteria 
prov?Lcled in the Wricer's Guide. The factors considered werc:: not intended 
to be exhaustive. In fact, in order to encourage their use, there was 
an active attempt to cainini^e factors. Additionally, only those factors 
that could be explicitly and objectively quantified were included. The 
intent was to remove subjective and arbitrary items from consideration. 
Material such as "figure title is descriptive" or "figure level of detail- 
adequate" were omitted due to their argumentative nature. The criteria 
scoring and their associated ranges were determined empirically through 
the use of a group of technical writers. These standards and their 
associated values have yet to be validated. However, the intent at this 
time is not to "legislate" the criteria but rather to provide to the 
technical writer and in-house quality assurance personnel an appreciation 
for- factors that contribute to use?.ble- technical manijals . 

Compiiter Simulation for Job P- rfomance Aids ■ 

As a first step in this effort a literature search on jot performance 
a.*:'.: (.^''o-.-.b. Cole, Sakala & Smi.llie, 197V' -vas reviewed in wro-c to 
obt..ln a workable operational def initio: various categorx'-^^j tional 

schemes. The current trend of where th- ^arch is being done and what 
kind o.c research it is was also assessed. i/inaily, the use of computer 
simulation as a tool for p:-;r f crmancc aid analysis was explored. 

■It was found that most of the job per f or;n.:.at:e aid researchers were 
active in the field for tr.:!/ -a limited period of time. Duriiig ^che. years 
1950 through 197;j, 78 percent of tac authors in the field ]-rorluced no 
more than a single contribution, Tihis means that the turnovc:r for the 
performance aid researchers was substantially high and thu.s c->. eluded 
formation of: an-- viable base of scientists for furthering the state of 
the art in this area. 

The growth rate in the state of the art was found to be rather slow 
and falls far below the rates for other research disciplines. For the 
years 1950 . through 1973 there was an increasing trend, with the peak 
occurring around 1969-71971,' One of the main reasons for this lack of^ 
exposure lies in the fact that the performance aid literature has mostly^-- 
appeared in government technical repot ts . This lack of circulation can 
be attributed to the fact that agency distribution lists are limited and 
not all libraries are govarnment depositories. Only 32 percent of the 
literature has appeared in scientific or technical journals-- 

A second step in this effort was the development and testing of a 
computerized algorithm for the analysis and appraisal of job performance 
aids. The proposed approach made use of the graphical evaluation and 
review technique (GFRT) syr.tem (Pritsker ^ Happ, 1966) > In a feasibility 
study., the basic concepts and hypotheses of the approach were shown to 
be adequate for analyzing and assessing job performance aids. 
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Using GERT-III Z (Pritsker Sigal, 1974) and Q-GERT (Pritsker, 1974) 
simulation packages,, a net^njorknodel was developed by (1) determining or 
defining the basic performance steps, (2) obtaining the time required for 
performance of each step by an integration of experimental data and 
measures obtained using predetermined time and motion system (MIM) , and 
(3) estimating the probability of error as-;«ociated with the performance 
of each 'step (data ol^tained from human factors literature). Once the- ■ , 
basic network was constructed, x^arious parf orma-.ice conditions and 
limitations or resources were*^ changed • The ::es'ilts yielded substantial 
statistical information on the effects of the various aid formats on 
performance time and errors (Smiliie, Edsallj, Sakala & Ayoub, 1975). 

These results were validated in a corap anion rj<perimental .study 
which evaluated four types of performance aid formats (Sakala, 1975). 
Tlie formats examined were (1) print; (2) pictorial, (3) prirt and 
pictorial combined, and (4) a hybrid group which used pictorial on easy 
steps and combined print and pictorial on the- more difficult steps. The 
experi.T.ental procedure consisted of fifty location and opo.ratiou tasks. 
The tasks consisted of movements of toggle switches, rotary switches, 
push buttons 5 ' pegs , and connectors. The subject viewed each instruction 
and then located and utirfonaed the indicated task. In addition to format, 
the task steps were divided in half to compare the performances of each 
set of twenty- five tai.;k stepsj i^e., allowing the subject :o 0^:1 ^ome 
familiar with the type 01 ins ur-zctional fcrriiat as well a:., the apparatus. 
Performance differences beaveen males and female::^ were also examined. 

Dependent measures used were time (i*e., c^jmbined time of reading 
the instruction plus per rl-jrinance of ins truct ion) ^ number of errors, and 
number of times subject reviewed izask steps*- The statistical analysis 
found less reviews for the .second half .of the task steps, For time, 
performances on the* second half of tlie task steps were faster but inter- 
acted v;ith format type. For the second half of the tasks the print format 
had the slowest time . v/hi I-- th$ combined format had the fastest. For • 
number of errors, perforr. .ace on the two i-i^ Ives were found to interact 
with sex. Females had more errors than males on the first half. Females 
were also' found to improve in performances, i.c*, fewer errors on the 
second half* . ' 

Future embellishments of the sir::ulation technique plan to use the 
statistical output to per corn a cost/benefit analysis. This would allow 
the determination of an optimran system x/nJch incorporates the minimum 
of performance times and errors under prescribed conditions. 

Format and Media Selection Technique 

As mentioned e.-irlier in this paper , i\\o, V,avy typically chooses to 
apply a standard f ormat/rnedia technique for all the technical documentation 
.for a given weapon system. Format in this conte:<t refers to the content 
and arrangement of information within a -presentation. Medium refers to • 
the means used * > record, store, or display tlie information. Generally 
the display .r;e of tlio. medium includes the senses tiif-> 'i:?r-r employs 

to perceive /.lie ,iisp^ay (hearing arAd/or vision). The staiid.ird f o.rmat/media 
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trtToTof Z t -f^-iencias it allows in the production 

tosts^.f . ! ^^"^ "^^"^^^ ^^'^ T"^^^ '"ini^^ design 

/W^! ^^'^^°"sjstent technics :nanual products are attained even vhen a 

'i'lorS^'i^ Of vendors contr.^une to the overall effort. The decision 
addlTnf "^^^^^^yly in system design. Media in general, until the 
advent of microform (film and f iche) , had, been limited to paper. 

Aa underlying premise of this effort is that the "standardized" 
technical manual approach, although the most efficient for production, 
A. ^H^^ t '° considered in selecting format and/or media. ' 

An additional factor is technical manual useabllity. ' ' 

proc^Jr'^'Th^f ^'^^':\^^^^^^ons are made early in the system acquisition 
process. These decisions are made by Navy officials az the project 
manag..T:ent office level or by designated technical documenta tioi repre- 
sentatives,. Equipment component engineers representing goveri-ipnt or 

organizations participate in these decisions. Contracts and 
budgeting constraints obviously are affected by these decis-^ons -nd 
once ma^e, severely linit the desrae' of freedora in later -.rc-.-a::^. .^ac.es. 
This ecfor.t..(Po3t, 1975) therefore undertook the task of uet^^n:.-- .■^^e^ f ' 
the selectiori -of format and media could occur before detailed dcsic-n " 
^"o c-r^-/'' ^"''m ■■^■^■^If'i"^^^ 'i^-iS^ data affords sufficient informa'tion 
y:^^!^ ""t ^ ^PP-'.f°^--^ ^-^d media approaches (prescriptions) 

baoea-on user data requirements (warrants). The first -ask was ro compile 
..nd categorize the systeir coriditioas winch have the potential to "affect 
tae format and media of a maintenance system. ' 

to aF^^^^T'^f^''°^'■'^'^ nineteen sys ten-conditions which have the potential 
to af.cct the format o.r media of a r^.aintenance information svstcm. These 

c^n^Ler.atlon:.'"^'^'''''"^'' equipment., personnel, and workspace 



Equipment 

Equipment Class 
(Types of Information) 

Equipment Class 
(Access Requirements) 

System Size 

Equip.-nent Distribution/ 
Layout 

Status Displays 

Support Equipment 



Personnel 
Time to Proficiency 

Personnel Turnover 

Span of Supervision 
Navy-Wide . E:^perienr.e 

Rating Selection Scores 
Safety 



Workspace 
Illumination 

Workspace 

Wind" 
Noise 

Dirt' 



Technician Res pen 



se 
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The study identified format and media features implied by each of 
' ^ the foregoing system conditions. Twenty-nine format and fourteen media x 
features were identified. 





Format 


Features 




- 

1. 


Procediiralized troubleshooting 


15. 


Tesc point .index 




aids 


16. 


Display index 


2. 


Deductive troubleshooting aids 


17 . 




3. 


Integrated troubleshooting aids 


1 

xo . 




4. 


System descriptions 




n plrri p f 1 Ti P cyin'Hp 


5. 


Readability enhancers 


20. 


SOPS for RTT cpt-nn 


6. 


BIT outconie index 




xTi-UL-caui. dx ±^ DXJL ubage 


7. 


Pocket-size format 


22. 


Locator pictorials by job 


8. 


Work, package 


23. 


Portability 


n 

J • 


Proceduralized aids for 
non- troubleshooting 


24 . 


General purpose locators 


10. 


Symp tom/cause tables 


25. 


Operator recording forms 


11. 


SOP descriptions 


26. 


S}Tuptom collection aid • 


12. 


Parts lis tings /identifiers 


27. 


Learning features 


13. 


Job index 


28. 


Job-relevani: graphics 




Symptom index 


29 . - 


Graphic density 




Media Features 




1. 


Centralized storage ^ 


8. 


Head/body-mounted " 'ewers 


2. 


Microform rnode 


9. 


Disposable (printouts) 


3.. 


Print mode 


10. 


Laminated materials 


4. 


a d ing portrayal 


11. 


Mini-viewers (hand-held) 


5. 


Ixe ^'Mi^g and listening por.trctyal 


12. 


Motion portrayal 


6. 


Self —illuminating devices 


13. 


Listening portrayal 


7. 


Large screen portrayril 




Hooded viewers 



Prescriptions, which are the application of specific format and, 
media alternatives to a system condition, were then developed. As an 
example, consider the condition of personnel turnover, which may cr.cur 
in either a high or low state. High turnover produces a work crew with 
a large percentage of inexperienced technicians. Generally, the 
troubleshooting aids typical o*f conventional manuals are too complex for 
, inexperienced technicians- The result is that a few experienced 
technicians carry the troubleshooting workload. The proceduralized 
. . troubleshooting aid is an innovative format developed to alleviate this 
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problem. Resaarcn has shown that inexperienced technicians can perform 
troubleshooting when supported by the proceduralized aid form. The use 
?t ^^Tf in this condition creates a potential for a more. equitabl( 

distribution of the troubleshooting workload. 

^ The eighteen system conditions involving 45 states and their associ- 
acea irormat aud media prescriptions were developed. The output of this 
ertort xs intended to be a guide. For ease of use the major problem or 
issues of presenting maintenance information were ased to organize the 
prescriptions into related sets. Specifically, the user of the gu\dei3 
astced to resolve six documentation issues hy considering relevant 
prescrip Lions. The documentation iss-aes are ?s follows: 
(1) Sy Stem description forraat.- 
. (2) Recording medium; 

(3) Access means; 

(4) Portrayal mode; 

(5) ^ Troubleshooting fotmat; 

(6) Workplace compatibility. 

The reniaiader of the guide discusses the si.x information issues and 
how relovaat prescriptions are used to resolve them. The discussion for 
each issue covers the following topics: 

(1) A statexaent of the issue; . • 

(2) A tabular summary of the formats and media prescribed 
for each relevant state, and; 

(3) Discussion of the rationale behind each prescription, 
including an illustration of the type of format or medium 
involved „ 
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SUMMARY 



The Navy is confronted today, more than ever before, with tx^70 
cardinal;.. facts.. of life. The first is that the processes used to ■ 
develop technical documentation - be it a technical manual, job aids, 
or whatevier - must be developed in a systematic and deliberate fashioa. 
Technical documentation is at a place similar to that of technical 
training some three to five years ago. In training, the Systems Approach 
to Training (SAT) and Instruction System Design (ISD) emerged. In the 
near future it will be necessary to have a documentation analog, Xi^hat- 
ever it comes tc be called. Three of the four efforts summarized in 
this report bear a direct resemblance to ISD/SAT processes. The 
Technical Writer ^s Guide calls for many of the same processes enumerated 
in handbooks for course developers: processes such as task identifi- 
cation, task analysis, and formative evaluation. The format/media 
selection effort is certainly seeking to create, an algorithm similar to 
the training media selection processes contained in all ISD/SAT models i 
The analogy with the job a i.d simulation techniques is likewise obvious • 

Tlie second fact of life is that there must be continual cross 
communication, fertilization, and a catalyst for the integration of 
relevant efforts in documentation research. Certainly the NTMS program 
described herein has this potential. Although the primary goal of the 
Department of Defease is to deploy effective x^eapon systems and not to 
train better, write better or read better, the to'tal system requires a 
delicate blend of both the machine and personnel components. The Navy 
has a strong vested interest in the communication of technical infor- 
mation - in communication for the purpose of training and also for the 
purpose of on-the-job performance. 
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\ . . ■ 

COMMENTS ON" THE Pa\PER BY MULLER ' 

-George Klare 
Ohio University 

Regarding Muller's paper, Oare con^aented on the: staggering effects 
of the large numbers and the costs involved in the Navy's technical . 
manual system. He suggested "that computer-based storage manuals, with 
terminals for retrieving the iixf ormation, might be vorthwhile to 
explore for the Navy. In this case^ research is needed on the effects 
of not being able to flip backward and for^^ard easily, and so on, on 
comprehension or other aspects of information processing. Klare noted 
an existing national library network called OCLC which currently pro- 
vides access to the holdings of many libraries; plans have also been 
made to use a communication satellite for more efficient dissemination ' 
in the future^ He suggested there may be value to the Navy in exploring 
this type of system for its appi;,cabxlity in the Navy technical manual 
system^ Finally, Klare pointed to the- need for further research to . 
better undeL\stand and iiaprove the useability of manuals. 
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PART III 
OBSERVATIONS AND SUGGESTIONS 
FROM THE CONFERENCE 
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irn-RODUCTION 



The Con^2Amcz on RoxidLyig and RoxidabiLUy ReASeoAc/i In thz 
^€AV^Cie4 served to bring together infonned military and civilian personnel 
to consider the past, present, and future course of military R&D in the 
areas of reading, readability, and useability. The question '^Where do we 
go from here?" underlaid the Conference discussions; -uggested answers to 
that question^ emerged from the major Conference activities: (1) the 
presentation of perspective papers by personnel repress en ting the Services 
and the response to those papers by the conferees, (2) the discussions of 
key issues in small Working Group sessions led by the civilian consultants, 
and (3) the general exchange of ideas among the conferees throughout the 
Conference, . 

Some recoimnendations for the general trend of future R^D in the 
Services have been presenrnd in the papers in Part I of this volume. Part 
II contains the perspect papers and* recommendations for future 
.ifesearch and actions. T; present section presents the observations 
and suggestions which resulted from the Working Group sessions in the Con- 
ference and from the statements delivered by the civilian Working Group 
leaders and by operational personnel in three Armed Services. All of 
these sources were, in turn, influenced by the informal interaction among 
conxerees in response to the perspective papers that were presented bv 
research personnel for and from each Service, 

Thv. source material for the. combined observations and suggestions 
consisted of thcv following items: 

1. Brief idea papers prepared by the civilian consultants, who had 
served as Working Group leaders. These papers presented the con-' 
sultants' impressions of the Conference and their recommendations 
for future research in reading and readability in the Armed Services, 

2. Transcripts of the recorded statements of research and development 
needs which the Working Group leaders reported to the general meeting 
on the last day of the Conference, 

3. Surmuaries of the Working Group sessions made by the HumRRO 
facilitator of each group. These summaries, of key issues and 
recommendations vere prepared in part by listening to the tape record- 
ings of the Working Group sessions, 

4. Traiujcripts of the recorded statements of research and develop- 
ment needs delivered on the last day of the Conference by operational 
personnel from each of the three Services, 

The starting point for the compilation of recommendations and suggest- 
ions was the idea papers by the civilian consultants. These papers were 
wrxtten after the Conference, allowing the ^ consultant? to summarize their 
xdeas after some reflection. The editors integrated these papers into one 
combxned paper which organized the various observations and suggestions into 
topic areas. 
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After this combined paper was prepared, .the Conference transcripts and 
summaries of Working Group sessions were carefully examined for implica- 
tions and recommendations for future research or vsuggasted actions. These 
were compared with those in the combined paper. Any points which were not 
in the paper were added in the ^appropriate section. Thus, the combined 
paper, which begins on the following page, brings together observations and 
suggestions from the entire Conference. 



I 
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OBSERVATIONS AND SUGGESTIONS FROM THE CONFERENCE 



GENERAL RESEARCH AND DEVELOPMENT CONSIDERATIONS 

Increased Coordination Among the Armed Services 

. There is a need for each of the Armed Services to expand the efforts 
to use what ths others^ have learned. One of the results of this effort: 
would develop s common data base and provide for regular meetings for repre- 
sentatives of the services to share practices, results of evaluations, and 
other research on their literacy programs. One mechanism for doing this 
would be an In'^erservice Committee. ' 

It would be helpful to have some agreement on carrying out cooperative 
btudies or at least agreement to use certain common tests for pre- and 
post-testing and to collect certain standard data on each of the men 
participating, in the literacy programs, 

Increased Interaction Among Civilians 

and Military Research ^and Development Groups 

Both the military and the civilian research and development groups 
could i^rof it f :ori increased collaboration* University-oriented research 
appears to be better organized in some areas than the mi.^tary's, but 
suffers: froc^. a lack of practical focus which the military coul(! provide. 
Similarly, the military literacy programs provide ideal laboratories for 
instructional research. Perhaps this conference will encourage the hiili- 
tary and clvili;in research and development groups to seek cooperative 
research and development efforts V7±th each other. 

RESEARCH Ofl BASIC SKILLS AND PERSONNEL CHARACTERISTICS 

Understanding Marginally Literate Personnel 

For developing programs that are effective, more needs to be Icnown 
about the men in each of the^ Armed Services who need literacy instruction. 
This wohld be particularly useful for those below a fourth-grade level and 
al^o those between grade levels 4 and 8. 

The causes of severe reading problems in adults could be investigated 
throu^f^h. an intensive study of 100 or so men selected randomly from those 
receiving literacy training in each of the Armed Services, such men to be 
given a thorough evaluation and diagnosis similar to one given to children 
and adolescents referred to a hospital or university clinic for reading 
and learning disability (dyslexia) . This intensive study should include 
an evaluation of the various components of reading and related language 
skills; (spelling, writing, auding) ; verbal and non-verbal intelligence 
(e.g., Wcchsler Adult Intelligence Scale, Raven's); abilities in other 
basic .-icademic skills, e.g., math; neurological and psychiatric examina- 
tions; and a case history and interview on early development, school 
history, and experiences. 
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Such studies would be of value generally, for in the civilian 
population more adults are presenting themselves for assistance as 
dyslexics. Thus, any study that might lead to a great^.r understanding 
of the cause(s) of severe reading disability among^ adults of normal or 
higher intelligence in the Armed Services would be helpful also to the 
general population. Indeed, studies of 'this kind on civilian populations 
selected to be roughly similar in socio-economic background to their 
counterparts in the Armed Services would be valuable in helping understand 
and cope with the literacy problem in the military. Such studies could 
provide each of the Armed Services with knowledge about personnel needing 
and getting literacy training. Specifically, they can help answer such 
qSfestions as the following: \^at percentage of the men heeding basic 
literacy training are essentially non-English speaking? What percent 
have a low ability on verbal tests? On performance tests? On both? 
\Vhat percent have a discrepancy of two or more years between reading 
achievement and mental ability? Ivhat percent are reading "up to their 
capacity"? VThat percent have medical problems — neurological , psychiatric, 
etc.? l-fnat percent had ^ poor schooling? X'faat combinations of these and 
other factors seem to be significant? 

Such information is needed for the group^ and for each individual in 
order to provide suitable remedial programs. It is needed for knowing 
hov/ best to teach them and group them, and what Ko expect from them in 
what amount of time. 

Decoding Skills 

The role of decoding in adult literacy instruction needs to be 
explored. VJhilc decoding appears to be important for children learning 
to read, if may not have the same value for adults who have al'ready been 
exposed^to initial reading instruction, but Iiave failed to reach the level 
of mastery raquired for job tasks. 

One approach to this area is to device (or borrow) a decoTiLng test, 
based on made-up (synthetic) words and administer it along with a general 
reading tefit to a cross-section of poor readers. By c^Kamining the rela- 
tionship between the two tests in detail, the scope of the problem might 
be seen. 

Comprehension Skills 

The question' of comprehension is not a simp''e one. Most people would 
agree that to use the term "comprehension" for^ the full range of skills we 
are concerned with is- to deny the complexity of the problem. On the one 
hand we have something we might call word meanings or understanding of 
vocabulary. On the other hand, we have the much more complex skills that 
are involved* with organi^sing complex relationships and with search 
strategies. Clearly, we need more work in conceptualizing the area of 
comprehension. 
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. Does a "comprehension" uuique ' to reading axisc? If not, then what 
has been labt.led as comprehend ion p; ;,l.-nis in Lhe past: needs to be re- 
evaluated. Oae hypothesis is -f.hat loi -;-rt:erials t^hich arc ' understood by 
listening, the only barrier to reading comprehension is speed of word 
recognition. Givftn a limited huinai. processing ::apacity, whatever 
capacity required fov irecogni-iing words is not available for comprehension. 
Tnxs hypothesis can be -and should be -tested empirically. Speed of recog- 
nition can be measured, as can listening and reading comprehension. The 
reiatioushi;) of speed of. r -.'cognition f. cowprehension ability can there- ' 
^ore oe determined for ..: vc-iriety of different types of materials. 

Cue factor T.hic- thi.-; livpochesis does not consider, however, except 
■xndiro.ci-.ly, is reading/prr,, e.ssing s .Trstei^y . A person who. has read a lot 

bat;t.L-- descriptions, for <';:-.mple , usually develops an ability to 
orgar.:.:^e the iaformation whxch he reads in a in.anr.ei. that: facilitates under- 
standing. A novice, on the other ....nd, is usually overwhelmed by unit 
numbers, attack directions, equipment lists, and command structures, even 
If his general reading ability is high. Further attention in both research 
ana training snould be given -o read:;.ng strategies _ that is, co how good 
readers organize aud recall couipj.ex inforaation in reading.' 

The Aiftomaticity of Basic Skills 

_ ^ _ l-uadaaentai to tha ::- .:odi^.g arui compr-hans ion .skills discussed above . 
IS ..ne importance of a uLgii degree ol: practice and overlearning on the 
S.C1I.S chat form the basis ?or higher level lenrning and compecence. 
.^.esearcti has pointed out that frenMently a significant difference between 
individuals who good and poor at reading comprehension is not neces- 

sarily skill : t ; ,cr levels uaderstanding , but r.pced and accuracy in 
simple wora decod:::g, a basic reading skill. .The necessity of spending 
additional ex.fort nnd us In;; additional menory capacity on very basic 
skills which st.ould be Derfor::,cd rapidly and autni.atically detracts from 
die abilxty to get meaning in the coui-;o of reading. 

Furthermore, as reading passages ge,;. more complex and as basic word 
decoding ski.Us are mastered, ■. there i.v the additional necessity for auto- 
matxcity to .'.ake p i.nce in terms of iiighe.- level units; i.e., in terms of 
syntaccical sentence units. ■ The inabili :y ■ to carry out these fundamental 
procejsei, rapidlv v/erloads an individual's cognitive capacity and 
detracts from the .-ipacity tu abstract meaning from a passage. The ■ 
situation is not unliKe expertis.; in a sr..-; Ixke tennis, where lack of 
mastery m the basic .strokes ln:erferas . ■ the abilitv to incorporate 
Higher order routines an-i strategies. The i.nd iv • d.ial ' s capacity is over- 
loaded with concentracion on the basic skills; development of automaticity 
m these skills would fxee h.Lm for higher level performance. Research on 
the nature and developmenc nf autoraacicity in retrieving the basic concepts 
and using skills thac uncierlic functional .joh literacy, general literacy, 
and_insti.tutLonal liccracy^- ,..:.uld see::, to h.. a research and development 
activity with high payoff.. 



^ These tenns are d.Lsc.us.scd .i.n the foil 



owji-,g section. 
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CONCEPTUALIZING THE NATURE. OF LITERACY- AND OTHER COMMUNICATION SKILLS 



Concepts of Literacy 

During the conference ^ an individual in the Armed Services was 
characterized in terms of three kinds of literacy: functional job 
literacy, general literacy^ and institutional literacy. Functional job 
literacy pertains directly to competence and expertise on a job. It 
involves .thc-i knowled.ge of important job concepts and the relationships 
between them, the ability to perform complex skills and routines, and. 
the ability to be able to uae and retrieve these knowledges and skills in 
appropriate situations in an efficient mannf»r. Functional job literacy 
can be significantly influenced by job performance aids of many different 
kinds. 

Significant quescioiis were raised in this area. Oii^j p^::r lairs to the 
relationship b>2t^*/een job expertise and abilities in verbal literncy skills. 
The strength of this relationship varies for different jobs, md might 
there not be a tendency to. "overliueracize" the requirement^i for particular 
jobs? Training may be d£:signed tso that it is more verbal th'ri:\ it ^.^hould 
be, since therrii is an instructional bias for teaching and tesc.iPLg in a 
verbal manner. Verbal teaching carries with it elements of prestige and 
of cen is easier to corry ouf. >. A second question pertaim-: ro ^h-^^ rc>':ogni~ 
tion that job expctr ti:-;i?. ii.Vviives the development of a network of co-.icepts 
in the iiiinJ of a cor:ipp,tc-n t individual. The question raised is how various 
job concepts are inrerrela -.od for the expert, and how this diiifers from 
the knowledge iitructures of the novice or the less competent i ndiv/idual. 
Current advanced research in cognitive psychology is beginning to develop 
v/ays of displaying "r.-:) reptual semantic networks," and the tc.chn [qiies of 
this kind of research r.iay he useful in describing liow knowledt'.v! i? orga- 
nized for the expert f^o that such knowledge relationships can b-; duplicated 
through appropriate t.r:rining and experience in less competent individuals. 
Similarly, longitudinal studies and contrastive analysis research on the 
functional job literacy porform.iuce of good and poor reader^^ may' /i".ld 
knowledge which could be Incorporated into appropriate Oinictionai jo; 
literacy" hrr'tin Lng E:)i: th-a {^oor readers. 

Cl'^nerai lihevao[j pertains essentially to the kind of literacy that is 
necessary in order to be able co enter training programs- lt':a(] iir:^, to 
functional job literacy. Tnis kind of literacy involve.^ basic in t'i^ilectual 
skills;, i.e., certain levels of competence in the basics of readinc;, writ- 
ing, arlrhmetic, and f aiKiamental cognitive skills like those rnea:?ut:ed by^. 
various aptitude and intelligence tests . ' The dovr lopment and eahuncement 
of general literacy skills can increase lateral ni: ' M.ity so tirva-. i.ndivid- 
uals are not restrict:ed to a few job specialti- .:aa nave a /.'■idvr choice 
of a V a i Lab 1 c. a r e e x\h or s c 1 1 o o 1 p o s s ib il i t ies . 

Tts^o approaclv:::^ to developing general literacy are posyi]:i,e. One 
approac]\ involves the dir-jcr: teaching of the skills ar.J compr tepa Lef» that 
are to l^e learned in elern'antary' school and junior high school. Thlh; is 
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simple enough to «tate, huz of tea ■ difficult to impiement , becau.^e of' 
various motivational factors or an individual's feaUng of not b-;.^- a^,ie 
barrwf -''T ''''^'''^^^S possibility for bro.akiuR down"these 

«n^> individual... in functional job training Dro.jrams Tvcst 

so taat they can learn by hands-on experience and with a mininiu-::; of v.-'^-.l 
superstructure This initial experience could provide individualr vi'n ' 
some feeling .of competence and the ability to carry out interest L-:p 
useful duties. Once an individual develops a basic set of job conro^ ] 
„ and related job skills, a knowledge and motivational structure v^ay'be'^ 
established that encourages and enables the individual to learn general 
literacy skills that- can permit him to advance further in his iob ' In 
cms vay, the general lir.eraov block might be broken because po-.s^ bn^" ries 
have t>::....n opened for vhich these skills seem highly roievani: and irn^o.-tant. 

'1^"'''"^ ^'^'i l-.ng~cerm research approach to enhaucir.- p,c,ne>:.i\ Ureraey 
sLilLs ia vo carry nut ba.sxc research on the cognitive procesr,Hs rhat 
unaer.ie the nature of intellir,.«ce and aptitude. At the nre.senr "riir- 
this .ond of research is not too abstract nor far-fetched.' Npw resevrrh 
endeavors are investigating the processes that underlie the rr>vohou.^tr{r- ' 
al.y measured encitie. Inle llige.nce and ^.ptitudc. -u.se cnf/ti'..; now 
are being interpreted in terms of cognitive informatioa process ^nr "si- 
that may be amenable to specific training. 

ImUUit-Lonal Zi-tn-vrr./ involves the information and --onora-t ka.--' 
required to get along ir. -.he military setting. This includes M-.- -s ; ; 
to understand regulationr. and to. participate successful.lv to t how " " ' 
activities which contribr.te to ono's advancement. 

Literacy, "Numeracy", and "Dexteracy" 

The emphasis of this Conference on reading and readabiliry Jr-.-.,i. to a 
concern about the "overlite.raclKing" of craining, even tl,ough tl.ero' is 
excellent work going on that attempts to decrease literacy requiremen.-.s in 
job manuals by including more pictures, diagrammatic explanations, etc 
However, It needs to be reiterated , that individuals can lea nv in d-:ffPrent 
ways and. that for some individuals, iuit.ial emphasis on verbal sklirmay 
no. be the avenue to further compct-ence. The road to competen.-n n:iph;- 
best be approached by init.Lally emphasizing hands-on dexterity bv ^;h u-. 
individuals can develop expertise up to a point which can then incorporate 
necessary verbal component... So; in addition to literacy, Glaser us ed the 
word "dexteracy" to empha..l:ce the. fact that literacy .in oar soclr.-v ...y be 
overemphasized to the extent that the dictionary does nor. contain ■"..;„.h 
words as ni'jneracy and dexi^sraau. 

'flie fact that individuals learn in different ways means that -esearch 
must seriously examine both the verbal and nerformance .■.-omponnnrs .'>!-" ' 
training and the particular strengths of individuals being i-r.-in-d an-' 
optimize the relation.ship., .between the two. There .1:, a nice ^.v.,:;.,!. '.^ 
Illustrate tills point th;:" vr.lr.ne.H to takin<>, advantape c,;" tii- .„., -; V...^, 
strengths of individuals .and .adjusting training ar.co-r!ln;.:lv . 'W:, 
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(Wing & Wallach, 1971) considered the characteristics ol students who 
apply for admission to college. Examination of the working:^ c£ an actual 
collega admissions coimnittfje indicated that the usual j^uocedure was to 
give- strong emphasis to standardized Scholastic Aptitude T^st scores and 
to high school academic achievement in judging the acceptability of 
applicants. Less recognition was given to outside-of-s Uool attainments, 
such as science projects or special employment. The college admission 
process, therefore, skimmed off the cream with regard to test scores and 
grades but failed to grant explicit recognition to individual attain.- 
ments which expressed themselves in nonacademic ways* A simulated 
admission procedure in which nonacademic accomplishments were considered 
as part of the college selection procedure resulted in nhc- fjelection of 
an admitted class whose membership differed substantially some 50 to 
60 percent from a clcso admitted by the traditional practi.co:^ . Further- 
niore, the hypothetical new class represented a wide spectruni. or. attain- 
ments and performance capabilities. This kind of study su;-;gPi-' t.s tciat 
selection procedures of the traditional variety tend to vcsLrict the 
possibility of adjusting to the performance capabilxu Los of iri 'J ' -^Iduals 
in order to maximize succe.si^: iw various occupations. If this :>.;or.der; 
range of capabilities were considered, educational content and teaching 
techniques would have to adjust to the specific real-world accomplishments 
that students could bring to bear on their schooling. 

Wliile this example, deals with a high level college population, the 
emphasis on adapting to special attainments derived from e.'ithor outidde 
the military services or in them is worth considering as one poss.uble way 
of adjusting to the capabilities of individuals as a basis for their 
further training. By virtue of particular experiences such no hands-on 
apprenticeships, individuals develop expertise from which tlu-y roald have 
been precluded by an overemphasis on **literacy'' as compared to "doxtcracy. 
An interesting research activity would be to apply the t.Tc.tLc cinployed by 
Wing and Wallach for a population like that 'entering the military services 
An important aspect to (•Qn<>ider in this regard is that people can develop 
higher levels of specific talent that may not be correlated with measures 
of general verbal literacy (IQ) , Excellent chess players, mech.mical and 
electronic experts, for exampl.c, can achieve competence through largely 
nonverbal routes, such as the visual modeling of' others' performance and 
practical experience , 

Measuring Literacy and Othor Communication Skills 

Measures need to be developed for the Armed Forces, based vot only on 
reading, but also ou v7ritLng. speaking, and listening. The lis coning 
scale is needed to detenniae whether a recruii: can cumprehund oral English 
sufficiently for a task; if he cannot, a literacy course would be a waste 
of time. This, in essence, is wliat the current auding-read j comparisons 
are aimed towards. (See Groff's paper in this volume.) 

Speaking and WTl..iii^^ are equally important for sone. mili.:.ary tasks, 
y e t s e 1 d o m r c c e iv e t h. e a t: t e a L i o c i wh ic h r e ad i ng does. A s c ; 1 1 * d c. v (; 1 n p ed 
for listening and t* en d 1 ? ) c o u ] d be extended, w i t h s o rr. ti, m( ^ d i i v c n ^ \ c : » , t o 
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cpeaking and writiu-. tl:,o>^c,illv, v.xnp.-ence in cl.^^,^ l.,ti...r tv^-. r, ■ 
xvxil be (or is) -on-r.Lder.;':! f.>r proiTOtion. 

. i>- should br, noLiid .:i.l.r,.;.-,dy f, fair -^firc^nici^i^. n,;M/ --..•rp i 

especially^ into che. Amy, do noc «peak English as' their .:a--i./.. ' uni^uiigo.. 
A rulL uuilization of i;innpowt-r ro.sourca requires dKir o^r,- '■m-Ti^'h 

be aases.sp.d and taught. l,i bailding such tests, i.t should be no ar-w nut 
tnat grade levo.: norms are not satisfactory criteria tor adult TU^cv-y 
goals; whether they are useful even at the eleaieutary level.; i:^ df-.ba-.nhle 
besides their arbitrariness, grade level norms are based unon vorabulary 
subject matter, and syntax considered appropriate for elementary children. 
To apply these to adults is in part inconsistent and in part degrading. A 
nore useful approach to setting reading criteria could be based u^)ou the ' 
Job Reading Task Tcots which ha..-e already been deveiioped for rSp Amy 
■ " f"'- S^als ot Armed ?on-.o.-. literacy programs are t..sk- o-ient.ul. fh-n 
task-oriented crxc-.r.a are t.ir more appropriate than criteria bar.e-J on 
general vocabulary and coir.prohension . 

LITERACY PROGRAMS AND TRAIMING SYSTEM CO.'lSIDERATiOriS 

The Study of LUer'acy Training Processes 

In the presentation nx training studies, much concern v;.is c;-.^;,,, ro 
•selection vari.abl , to rV.e ra.vsurameii t oi output achieve.;:... r,. ■■ ,•: ■ -be 
measurement of slior;. • or lonj-tem, ofTects of trainitu^ on :ph ' ;r .-'^,-.;i;.ace 
Loss detailed atranl,;l-n. :;..s ;;l-/ea no -hat actually haopei.ed da- .- :-:,ia- 
-lag. There is obvious attention to the development of Lra:>.;.;v n'-icVn,-c-.s 
and to the comparison of different forms of training. Howe-er ' "Uti-le 
study seem^ to be made cvf tl.e details Of actual trainlnf; imp! ,-n..n:-at j'ons 
For exanple, when train in,;, nerhod A compared with trnMi In- ''icthod •■; 
the programs: are de;-crj.I>ed only in terms of gross eomponenr.; ■ \ivz\^ infor- 
mation IS provided about che details of hov tiiey are actual! v i r.nl em-nted 
and about the dlLaen.sious of eadi that were successful or thu" d imruislons 
that vere unsuccessiul . It h; difticult to compare brand X a;;aiasr. bran^ Y 
primarily oy thc-r labei.s-. It is necessary to know v/nether artua. tr.--'- ft.- 
conditions were letter rcr one than the other, whether field ron^'^ilc 
reduces one to be in- th,: ,san:c a-, the other, v/hotaer one inv,.,- L .'ad a bat; -r 
group of instructora, or wuetiier one permitted more time to atady, -tc. 

^Wfiat is requirt:d in aaseras i a?.; various training proc. -.aiuras la :.h,- 
de taxied analysia of. claii;:,rnr,n arocesses .-id implomentat ion r.r» 
'Oiis require:: i.^olation and obs -.r'/ation of a number n ~- n-V" . n■..-a^T■ • a-, 
proccs.ses that include a,.a;! dimensions as (a) oDnc.--aav: -a,' -a-rin 
the tirae available for actual on- task Involvement; (b) mr tch i nr - che 
rclatxo.'i.ship between what is acfially taught and what Ir, aca;. T:v resrad- 
(c) material motivati,.,a - ..ht ,;,>^ree of Interest iai i 1 r : nm - " ^-^ ; ■, ; ' 
m.iiterials; (d) teaci.liv- moriv.ition - the d.-:ae of exci fenea- .-.a-e-d d 
by the ,nstractor; (e) s:-ac:ara .-.ad, seCi.-.ecc - the eaan-- ta -./■a'-a a 
curriculum is carefMi;., ;;aqaeacaa in a form of a hierarchv aT n a ^ s i .-s 

ther.a la r,p- tun f ry f.a- mastery of earlier akid.,;:; 



to more conpiex skills; nnd f) monitoring and feedback --- the degree 
to which the training pioccb ^allows for Lh^i detciiled an.il>v3lG of scudent 
strengths and w^aknes'i^es iu che. course of learning, and how i. ;:ructio:. 
is adjusted to this infoanacloa. An inceresting research endt^avor ±3 zbv: 
conduct of field studies Lha': investigate x/hat is actually t:auf;hc and 
tested, and how it is taught, so that processes of traLniii^j; can be Inv- 
proyed and so that differenic training systems can be given' adequate cc^x- 
parison. A trainincv system r.ay also be compared to a theoretical c.'!:'.<:o-- 
tion of the expected .cnange i-i the reader to determine wn.iither the ; .in- 
ing has been effective. 

There is need for a study of the teachers of the literacy prog.-:.as 
to determine which cDnpeteacies are necessary, how thev L.av bo. dev--:.ped, 
and how they can be evaluated. 

The amount: of t.^r/.e ucrcessary to effect a significan.: imoroven. - .: in ' 
reading achievement needs study. At the Conference, it wai; I'n.ide clc .r 
that only -a limited ainouat of time has' been scheduled fo-r- li:-.crr;.^c-/ train- 
ing, even for t:he men who corae with no reading ability wiuii:::iue v. r. is 
suggested thai: the u,>i.e:r:, j. ^ rather disappointing outcome.; of i\ ....... 

literacy training programs may couie from expecting thr- ii:ipossl-;c. . .e. , 
expecting 4 or 8 years of pr')oress , in terms of standard L2:ed r> . ;-: a;; 
achicvr-nont tests, in 8 v/erks jr so. Even if some of \ho. .ik iT : s 

can be acquired in so f^hort a time, it is ques tionab ; wherh^r thcj : it 
gration of these skills, can b:- acquired in this Vmi'O. l/ha :: i- iv ^-d uv 
children (and it .is suage.-ir.od that illiterate adults do no^: differ . nb- 
stantially in this respc'cL from children with similar readmr, levels) i. 
extensive reading; ana y^ractice over an exteivle'.^ period of ti-iU. Hov low.' 
a tiKK: J?; -oaded for adiilLs lo reach certain reading ] eve J 3 ..h:i:.i.d 
;j;vr./. i.^-.:r.-d. Snecifi: '.y , i should be 'determined wher.]K:v tl.c 'jni..>mt 
and t-ini.i;^, of the: ins r.ruc uion and ^.ractice varies with the eiUrrln:' r. -d- 
ing level, mental ability, and the kind of reading program u:>.;J :.r '"tea.'h 
the beginning fr'tadin;:.; skills. 

Studv is needed not only on the amount of time necess-'V' o effect 
significant i:ap rovcr^ent in reading achievement, buf: also on .he amount 
and nature of improvement: that: qualifie as "signif icant'\ .lar: cioes a 
given reading gaii: mean in terms of job per f onnc^nce ? Just as >i;rade levr^l 
norms are unsatisfactory criteria for adult literacy goals, they are un- 
satisfactory in rvaiaaMn- thu effectiveness of literacy ?:rHinLny prc;grams. 

Studies of Alternative delivery Systems 

Regarding the uO.! i very o^ li^:eraey training prcr-;-^;.. , ;.; would be 
well to look in'-o ihv n.-a-iibiliuv of making greater ^.jn .d La - TV, 

films, rapes or.d LeMt;s, cue. Wh j le there is seriouh^ tr::^- ^:nd-jrwav for 
producing such teaching nd ^tl : , It is suggested nha' :lr- : :::^nn;; learned 
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from such programs as Ch.il.dreu' s Television Workshop's Electru-. Conn...nnv 
be examined xor pcsoible use in teaching basic reading skill:, to men at . 
the iovesc 1-vei of readia- achi-vement. 

Cdnipuc«r-aided inr; traction is still in -an early experimenr;.]. stage 
so far as its capability to instruct is concerned. ?K.verl:hel...ss . for the 

• available time for instruction is sevorelv ' limited 

CAI.mzgnt be beneficial for sor.e types of instruction. It wouUl be less' 
sensativa to manpower ch^^Vge chau the present instr .. ^1 u .1 svst.T.- and 
^ed?i^''''^°^^'' ^-^'^^°^^P^' '-^"y oth'ir passive 

_ In addition, CM might b« used for continuing educnMon ou a f roe- 
choice basxs, thus allowing rai:iirar>- personnel the oppor : :-.n i 
courses on o^t: tin:e. TI,.. ''L.^ro project at the Ua;i v.rs. of Illi- 

noxs iuis developed supplemental reading instruction for LiM. prlr.,rv grades, 
Sorns o]-_vne:.r exnerience. misht be transferable to an aduit ...^t^n^. The 
Laivers.ity of Wisrconsin is e:.iperii..enting with the use of p;,.-, 



line interactive system t.cr diagnosing reading skill dp f - -"he -sys- 

tem has a complete, test analysis subsystem built in so Uutt r4, r-u./ts 
car be printed within minutes after the last person is tested. Orher 
ar.p..icavxoru7 of CAI , par:. ;.cul:n:lv those vhich provide iob-f,.Tif.--.-.rv- lit- 
e. .cy trainin-, shovild bp cr.n:;i>Jurcd . 

Illr££ri}±£iLl!itejja^^^ Cjil Traivvi ne 

All of clu-. Arraad Services r.hare the need of having nun, „, c^crvx v.-ry 
,specx..ic occupation:, ^.idi high degrees of proficiency. '„n,en r..M, do not ■ 
perform t.ho..r jobs or cannot learn to perform their iobs, thp.rc: : a prob- 
lem ot occnpa..xonal con.pc.tv.nuc . Literacy is one importaa;. ,,in.in. . but not 
the on.y tactor of proflclenc job perf onnance . In the Armed fnu-vLres 
which define their n>is3ion norc in terms oC specific job traLri n.' t-'.aa 
general literacy training, the extensive investment in job p.-r rop-i.mrP 
aids shows an i-nplicit recognition that the issue 1.-: Dr i.rrar! 1 v .-.n- oT 
occupatior.a'l competence. 

The humanistic value of i.iteracy training may also v^d-MMnnd as 
a neea .or uarnc coiap^-concy -tr.-ining. To effectively tranr-.u r J .-.-.racy 
to marginally 'iterate men, it: will be necessary to olace ce.:.ding'in the 
context 01 a .L..rger s.-t of cur.netencies . Essential basic compe.Lraci^^s 
for adults incl„de reading, writing, speaking, listening, calculatnn;' 
and probLer,-s.uv.w:,.. i. i,, rc...nable to suppose, taat thes.. , ■ --a:- 1 es 
are connec t.vu and that imp rovc-^en ' 1:: one will not be useful ,1 .^L 

tor tae i.na.i.vidua i. without cor, ::o;r.itaP.t improvements .in the qHi— « Ti.^t 
arc the listening, speaking, rea.iing, writing, computing, .rabi.^n,- 
soLvmg competencj.e- required for a given occupation? To • .-; of 
tencxes may not ba comploieiy appropriate for the tyo-s of iob..- i hat occur 
frequent.Ly In tho Armed Sc-rvices. However, it is cloar taat i n-. ; ech- 

nical documents would be one major aspect for a large pronort --...a of"'"i6bs 
Another major ar;pert would probably turn out to be u rob 1 n„ 'indeed 

proDlen,-soJving wi written do.ai.-^ents was mentioned rrn.,..--- . - ' th- ' 
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Conference. Good job performance is associated with frequent reference 
to reading matters. A cause-and-ef f ect relationship cannot be inferred 
here, since high intelligence could mediate between' the two variables, 
but the relationship is suggestive. In fact, visual search of written 
documents seems to be a relatively frequent and relatively difficult, 
aspect of job performance. There is, of course, a heavy technical knowl- 
edge load in many jobs. . ■ ... ^ 

It may be instructive to consider the arialogy between literacy train- 
ing m the Armed Services and literacy training in developing nations. 
Of course, there are points of difference, but the points of similarity 
are useful. First, it should be noted that, as reported in the 18th ses- 
sion of the General Conference of UNESCO in Paris, 1974, there is a sub- 
stantial undertaking known as "The Experimental World Literacy Program 
and Its Global Evaluation". Over a five-year period from 1967 to 1972, i 
the program reached more than one million people (1,028,381) in over 20 
thousand (20,379) instructional classes and employed a total of over 23 
thousand (23,480) instructors at a total cost of about 27 ^million dollars. 
V/hile the programs in the countries vary, they all draw a sharp distinc- 
tion between "traditional literacy" and "functional literacy". Tradi- 
tional literacy programs have defined literacy as reading at a given 
reading grade level (i.e., reading at the 4th- to 6th-grade level) and 
the training programs attempt to bring adults up to that level of general 
reading ability. For many, varied reasons there has been a widespread 
disenchantment with this conceptualization of literacy. For example, 
4th- to 6th-grade reading level did not provide immediate social and 'eco- 
nomic returns to people. Low motivation and inadequatelv prepared teach- 
ers contributed to attrition -ates that were so high that teaching could 
not be effective. In it^ place, the concept of "functional literacy" has 
been substituted. This concept embraces reading, coraputation , and knowl- 
edge in a specific area of social value, usually agriculture. For example, 
reading, computation, and knowledge about modern cottoii-growing practices 
are taught simultaneously in Tanzania. This type of program reflects a 
more complete and realistic occupational analysis than former programs 
and it is vastly more successful. 

At the UNESCO conference mentioned previously, the following prin- 
ciples of literacy training were tentatively for^jarded, based on experience 
in these projects: 

1. Functional literacy programs should be geared to specific 
objectives, problems or centers of interest. 

2. There should be an integration of various components (lan- 
guage, mathematics, technology, scientific acculturation, 
scientific training, graphic representation) into a unified 
educational process. 

3. Programs should be adapt'jd to socio-economic and geographi- 
cal conditions. 
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4. Teaching should be intense enough to produce desired 
results in a minimiiiD time period. 

5. There should be constant adaptation of programs and teach- 
ing methods to the changing technology and demands of 
development. 

6. Literacy training should be regarded as the first stage 
in a process of lifelong education. 

mile the populations in developing countries and the Armed Services' 
vary m many ways, ir. is ncteworthy that in both cases occupational com- 
petence is a major goal. In developing countries, it has been found to 
be imperative to train competencies such as mathematics and basic knowl- 
edge alongside the literacy skills of reading and writing if the goals 
of economic productivity and social advancement are to be achieved effi- 
ciently. PerhaRS the same principle would apply to the Armed Services 
in the United S.tates, 

One of the problems with general literacy training prograiua is that 
the programs do not improve a man's reading ability to the level neces- 
sary for functional use of the reading ability in job performance. "Increas- 
ing a man s skill from 4th-grade to 6th-grade reading level is valuable, 
but it is not functional for many jobs. In a functional literacy program, 
men would be trained to liiarn to perform specific reading tasks to the 
level required for adequate job performance.. Job technical- knowledge , 
concepts, and skills training would he ingegrated V7ith literacy training 
including job reading tasks; e.g., reading orders, searching the manual 
of inventory, calculating prices, etc. Men could stay in a training 
course until such time as they passed a proficiency examination with 
criterion-referenced, job-related reading task tests. Thus, men who 
needed to learn a number of basic operations, such as reading and writing, 
would probably stay in the course longer than men who were higher in these 
competency areas. This suggests a deviation from some of the current 
practices in the Armed Services. At present, in many instances, the inten- 
tion is that men will first be trained to read and will then be assigned 
to job skills training. This adds to the total training time and fails 
to capitalize on the immediate transfer to be gained from integrated, 
functional literacy and occupational training. One issue that is raised 
by such a practice is the extent of transfer of functional literacy train- 
ing to general, life situations. Research could be performed to explore 
the generalizability of job-specific literacy training. 

The overall ^rust of the notions expressed above, particularly those 
that pertain to t!i^ improvement of functional literacy and to adjusting 
job skills training to the developed capabilities of individuals, suggest 
a model of training that is different from the model generally employed in 
the military and in most civilian training and education. The usual train- 
ing system is a ^eZzctcvz system which emphasizes the selection of indi- 
viduals on the basis of preliminary screening tests that predict who has 
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a reasonable probability of s.ucceeding in a particular systeui oi crc.i.alng. 
^fhe system of training is set 'up for those individuals with specxfic enter- 
ing credentials and is improved by tightening credentials, but litcie ad- 
justment is made f.or individuals who might profit from extensive prerequi- 
site training in. order to get through the system or by change? in L'ue 
training. system itself to enable them to attain the full range of compe- 
tencies that are required and taught by the training system/' IncL-iing 
literacy skills. Research is needed to explore the feasibility of devel- 
oping the more adaptivd training programs needed to provide the full range 
of competencies personnel may need to have developed in order., to "-.access- 
fully acquire job knowledges and skills. 

Research might also explore the feasibility of extending.: such an 
adaptive training prograia to include pre-service functiona] lit.-.ra-."' rirain- 
ing in broad occupational areas. Policy decisions permitting, such train- 
ing might be offered by the Department of Defense acting for aJ.l th-.: Armed 
Services or coordinated v/ith other governmental agencies concern- ' i;i th 
adult basic education in job-related contexts, such as the Dep.-;- 1..-. -nr. of 
Labor. and the Office of Education. ' ' 

RFSEARCH ON THE RflADABILITV a USEABILITY OF MATERIALS 
Research to Improve the 

Predictive Efficiency of Readability Formulas' 

^S££L*l[!l£L^\!;_^iLil^^^ of Terms 

Such tcrmrs as "r-adahle", "readability", "comprehension", 'vnr. -hon- 

Sibility", and "u.soabili Ly" shoii.ld be reviewed and standard ck- f i ru t ■ -ns 
suggested, for Anried Sorvices useage. . Standard definitions wou.wl provont 



misunderstanding such as apparent differences in data which ar^i 
differcinces in Intorpretaficm. stemming from differences in r:n }xr. . 
The literature on readabllLty provides .a number of exauiples of di.::....;'r'i. - 
ment in interpr^i- Latior. that, on closer inspection, become merely 
sions ia usea^^' . As a cas-in point, the term "high rcadabllxtv :.:i-..,r,.'' 
may refer to som.^. tin' that is easy to read or something that iiar.-j to 
read. The reason is that with some formulas (e.g., the coir.nionly k-.-o 
Fiesch Reading, Ens^t [ornuia) a high score means easy reading-, vhi'y -^i; 
others (e.;-., the conuaonly used Dale-Chall formula) a high score i:..-,in3 
hard reading. Examples could be provided for the otlier terms as -.■j'-I]. 
Standard definitions might not only prevent unnecessary conf Lisio:i.s :,ud 
disagreements in Armed Services publications and useage, hi 
fully) spread tn educational publications and useage as 
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Criteria for Pr edicting Difficulty of 
Materials (Readability and Useabillty) 

_ A problem discussed at the Conference was the tendency tc talt" .-bout 
written materials as though .they were all similar in structure srJ pur- 
pose. A useful distinction was suggested between "operational" r-anua-is 
or ones that tell how to perform job tasks, versus training manuals. ' 
which are to permit the leamin^g of job information. Though one mU-nt: ' 
wish to use a readability formula on both types of manuals, the desired 
criterion for operational manuals might be task performance, while the 
use ot recall, recognition, or some other learning measures might be best 
tor evaluating the readability (leamability) of the training manuals. 
With regard to operational manuals, the value of the document might be 1 1» 

assessed by measuring Che time required t perform the task with and T,;ith- 
out the document within a defined pppulaLon of men (with general apfi- 
tude, p, and job experience, q) . Thus, the value of the document could 
be determxned by the ratio of the "with" and "without" perf orr.ancas . 
Sucn a criterion could be applied to schematics and reference pnnua-, 
as weii as books or other technical writings. 

Such a criterion would be useful in attempting to improve cerOLn 
documents. _ For example, the extent to which a manual is well nrrmri.-d 
and lends itself to search operations could be determined by exa-ijr 
the amount of time required to locate information in the docur.ien'- '! " 
quality of the document could be systematically improved by m.ininular 
Its characteristics in such a manner as to reduce the time needed For 
completing search tasks using it. 

Regarding the use of learning measures to evaluate the re.-.dni) i ■ ■• ry 
(leamability) of training manuals, it was suggested that su^h r.-a-M- s 
might be an intermediate evaluation step in the case of narrative rri^-ri- 
als used for training, ^./here understanding of principles is highly de- 
sirable, and that some kind of task performance might be a final n„trome 
.for training manuals, as well as for operational manuals. Anothp.r sug- 
gestion IS to use as an intermediate evaluation step, a "levels-of- 
comprehension" distinction, with a deeper' level (e.g., one that permits 
more transfer of training) desired in the case of training than in rhp' 
case of operational use. Differences among the Armed Services on Mvis 
question might also be explored. 

In, addition to possible differences in criteria to be predirt-l by 
readability f ormulas in the case of operational versus training i.ana Hs 
criteria for comprehensibility" and for "useabilitv" should be rPvi-./.^d 
and nhe value of the two concepts compared for Armed Services useage. 
_ The term "useabillty", though uncommon in educational publications 
IS becof.uAg comjnon in Armed Services circles. Criteria for r.- surinc- ' 
useabLlity, however, are not commonly agreed upon, and should be r-viowPd 
The term comprehensibility", on the other hand, is common enou;^h L-,-' 
the criteria used for measuring it give rather strikingly difenrr-nt 
suits. A p-ime example is the use of multiple-choice versu.s r]o--> r-..- 
prehension jicores. The former measure typically yields higher p.-r r.-. taee 
correct values than the latter, but validity coefficients for the ..-^^n'. 
readability formulas typically yield higher values for the latter ih,,n 
the former. 
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^gg^^H£L.oiLih£.^n dicat\rs of Re adahn^ 

• wholly satisfactory. Further Sve^t17^ f u'^""^^^ material) are not 
for readability indices mVht ITr . °^ psychological basis 
for the processing of ex 12 e^e^rged'^d h '""'^^ ^-^-'-'^ing .odels 
sufficiently well to warrant WDW^f , withstood empirical tests 
-dels to practical dl^^en LSon ^ork"' To^e °" ^^^]^<^^^^ons of these 
to assure that what is done iS^,^! ?" ''^''^ indicated, however, ' 

^ >ideas, but also by a reaSst^c'S? '^^^^^ scientific ' 

applied in practical seSngs ^(117? "^ °' realistically 
discussion of the difficSty ^f Vo^n ^"^^^ V^^^ ^^^^ for a 

sus increase in predictive powe.AP ''" ' readability formula ver- 

are p^ed-Jjiv^: f o^^^h^g^rte^^n^'cS^^^osl^^^l^^^^^^ readability formulas • 
A grid was suggested that ha^ Snds'1 material. 

• application situations along tL otl^ with p''' """^"'""^ "^^^ 
existing data located at the inLrsedM ^""^'"^^^ based on 
not exist, it would be immediatPl^ ^ °f ^'^^^^ ^^^^a do 
were needed, a la clTss^TellotlT^^^^^^^ -"^i- 
to suggest that the degree of IZr^ \ chemxscry. This is not 

. but the procedure .ouK s^in be usIfuA" M h"T ^^-^-^-y. 
to have all of the empirical data h^S^ Although it would be desirable 

example, a first apprSl^tLn^: u d^lt^Lm^d^tV-^ ^-^^T'^' 
it might be desirable to. have an indp^nf ^^ ^ Similarly, though 

writing for descriptive purposes cruL technicality of 

the first approximation.' In ime ^, ^"f ^--^^ ^^ht be sufficient for 

■ .ore complex grid and/or other^^^ids'mSh^t TZ^^^^' '^-'^'^'^ ^ 

thoughSulL''Slf:r'o^; J^eh^ch'a"' f'^'^ characteristics, should be 
free. Since, word difficuLy o1 word^^ H '''''' '-^^^ 

tence complexity in sot clses) are tL ' i -""^th (or sen- 

reader;s knowledge of conte^^rLrrbe^^p'rals^'r':^^ 'T'""'' °^ ^ 

extensive use of technical terms and the'iSor 't° " T-- 

^hfu^Hfl -sl:"'^ — - -^n^-^otrou^f JhJ^S 

writtS'\^;.t\^Sr^^s::e\lf -^a'ge^S^o^^ °^ — - of 

dicator of the readability of written do ^ i""P°^tant in- 

and form. Yet the reports real 't tL ? T ""^^^^'^^^y ^^yle 

of the afternoon discLsiL) LdiclLd "trna'p^r T'' '""^ ^^^^st.nce 
for the systematic assessmen of content .^p ' . ''''' 
addition to assessment in terms of Thp\ f rudimentary present. In 
cope With, the development of :ffecSve :eTh H'%'"^^: " ^'^^'^^^ 

i^hi-rs: --i- Jr:;ffiS:;t5 ^:;t;e'-:f-c:^-^-. 

practical_rese^rch^:;:jL^rrthr:°ii^---.-S^/^ "^^''^^ ^ -^"^ 
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One approach suggesteu for taking content into account in readability 
. formulas is to- cojnsiaer the technical teminology of . manuals as a separate 
factor in predicting readability. This general kind of question - taking 
special terminology into account to improve formula predictiveness fr- has 
bp^a around for some time. With °a word-list type of formula, this Ian be 
doue by adding v^cTds to the list or by excluding certain words ^Mom a 
count. With a. word-length type of formula, this might also be done by 
excluding v^^ords from the length count during application, but this can 
cause considerable inconvenience and, in the case of human (as opposed to 
computer) application, prcibable unreliability of application. A better 
possibility would seem to be the use of a correction factor based upon the 
technicality of the material, A technicality index might be developed for 
this purpose, using (for example) ' number of technical terms in a sample of 
text ^over number of total v/ordf? , or, preferably, over number of content 
(as dpposed lo structure or function) words. The usefulness of these ap**^ 
p^o.-ches is, of course, an empirical question, and studies should be con- 
.i.cled even though (or perhaps because) their success is by no me.ans 
a-sured. Trying to -.dd anothc-: index variable to fonnulas or to correct 
existing index variables m:-y unfortunately be less effective- than we 
might hope; adding an adjustment for technicality of the material may pro- 
duce no increase ..n the p . -.die triveness of the formula. But agalu, the 
question is an enipir.;. c:ai jiie muJ should be investigated. 

Research to Improve the r-i\jCL'Ci:ion of hiaterials 

Re search on the Selecti^n]__jui^ 'j.' rai-iu^ of Tec hnical Writers 

. While read;ibJ.].i.ty fornial;is are somev./hat efficient in. p/i^cUcJXHQ the 
reading difficulty level oF materials, they do not offer guidance for p/io- 
duc^ng more effective, readable materials. There is a need for research 
to help train writers to produce more readable/useable materials.- One 
approach suggested is to identify the better performers among groups of 
x^nriters in terms of their "track record" in producing good technical text 
over a period of tine. A p06t koc analysis might "then help eventually to 
answer such long-standing questions a- whether it is better to teach a 
technically trained person to improve h'j or her writing skills or to 
teach a trained writer enough of the n-.eded technology to permit him or 
her to write accurate technical documeniis. Even if one answer -.Jill not 
fit all -situations , this approach might help to say something about when 
one or .the other possible sources of writers should be considered.' 

If the characteristics of good technical writers can be identified, 
further work on the development of sclectior^instruments should be . under- 
taken-. Discussion indicated that many companies that produce materials 
for the military do not keep any ^certain number 'of technical v/riters under 
employ at all times, or at least do not keep those they have at technical 
writing tasks all of the time. Consequently, some selection procedure, 
could be helpful during crests in the need for technical writers. If a 
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test-type instrument proves difficult t*.. duv^ top) a prof i].e-type approach 
might be considered, making us^. of suca Vrvriables as background, training, 
experience, interest, etc. 

For training purposes, research is needed on what to tell writers 
x-jho wish to improve the effectiveness of th^^ir writing or who, alterna- 
tively, have been told their writing is inadequate and must be changed. 
Discussion suggested that writers seldom, if ever, set out to write less 
effectively than they can. Yet. this happens, and most writers sincerely 
welcome really useful suggestions for improvement, ^lat suggestions can 
be given? Writers* guides fail to agree, and in some cases ^ actually dis- 
agree, on what should be done in x>rriting clearly- One approach is the 
Army writing guide that i-. soon to be published, ^(See Kern's paper in 
this volume.) Kern found that Army writers seem to "model" their docu- 
ments after those that have gone before, and this guide suggests prefer- 
able alternative models. A second writers' guide produced recently by 
Klare presents suggestions fo-^ readable writing based upon psycholinguis- 
.tic and readability research. The effectiveness of both of these guides, 
as well as of others, needs to hz systematically evaluated. Such evalu- 
ation should include attitudes cf writers toward such guides, probability 
of use on the job-, et cetera. 

Writers' 'guidjes sho uxd also deal with the treatment of graphic mate- 
rials, for the worlk of technical writers involves decisions affecting the 
design of graphic hs well as textual material. Suggested research and 
development aetivihies include the following: 

a. ^Research land develop guidelines for the use of graphic 
materialsl Tliese guidt^lines should include the various 
types of A;raphic displays, as well as "best" type of 
display for ..a particular purpose.. 

b. Research akd develop guidelines on the readability of 
graphic materials. Specifically, x>7hat element^ in a 
particular type of graphic display make the dislplay more 
understandable? Also, what are the functions of graphic 
displays and\how can they best be utilized for teaching 
and /or learnmg? \ 

c. Research and develop guidelines concerning design dif- 
- ferences for materials which are to aid people in per- 
forming a taskWersus materials which are to aid people 
in learning a task. 
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Research to Provi de TeclLciic al Writers with 
Information About Reader Charact£:ristics and Purposes 

A need exists to identify the characteristics and purposes of intended 
readers of different types of technical writing, and to get useful informa- 
tion of this sort to writers. Specific information about the. 'important 
characteristics and purposes of readers of different types of material 
often. does not get to writers before they begin their tasks; when any in- 
formation is provided at all. It is likely to be too general to be of much 
help. Some wricers niay not even know what kind of information about read- 
ies could help them, and what kind they muZd lU^ if it were available to 
whem. 

^To provide concrete guidance to wricers, we need to V^ow a.^ and wk^n 
rewriting can make a rciirible difference in reader comprehension and per- 
formance.. Put cuiother way, a model is needed that takes inr.o account the 
several categories of factors that can influence the ef f ec tivonof^s of re- 
written material. Character istics of written material and of readers are 
clearly important: oharacc.-ris tics of the situation under which the mate- 
rial is used and/or rested may have more subtle but equally important 
effects. One such way is by influencing reader motivation and giving or 
not giviug;it a chance to aff.ec- x-oader behavior (e.g., through 'orovision 
of liberal reading and/or ^Of:tLp.g time). One prediction from\hli-; model 
tnat has been successfully tested, is that improved readability will pro- 
duce relatively greater iafonr.ation gain from a less preferred than from 
a more preferred content:. A namber of implications are suggested b/ snch 
a finding. For exa^nples , priority might well be given to: rtiwriting low- 
interest (versus high-interest) contents; putting the better writers to 
work on low-in teres L con tents; rev/riting material for low-motivation jobs 
and for low-motivation readers; most careful rewriting of the first parts 
of courses; et cetera. Further work on such a model might well be of val- 
ue for rewriting tasks in the Armed Services, since the staggering volume 
of technical material makes essential a priority approach to. the problem. 

Research on Methods for 

Assisting the Writer, As Writer and As Editor 

Discussion suggested that technical manual production, and particu- 
laily rewriting, are low priority activiii c:^ . The' growing power", and the 
attendant willingness, to reject unsatisfactory technical material should 
improve matters, since contractors must give increasing attention to tech- 
nical manual production under these circumstances. But even then, few 
who have done much rewriting would suggest that it is as rev;arding as 
writing an original draft. The computer seems to offer potential help, 
especially if the functions assigned it are those the writer does not like 
to do, or do well, and that are compatible with computer capabilities. 
For example, computers can be made to do language recording and storage 
tasks easily and efficiently; but language decision tasks only with diffi-- 
culty. and poorly, as the iriachin- translation literature' clearly r.hows. 
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Humans, on the other hand, have just the opposite proclivities. Conse- 
quently, computer aid to the x^lter should focus on the former, and should 
have the added characteristics of mapping onto printing and/or publishing 
functions already being done by computer or slated for computer handling 
later. For example, more and more typesetting is being done by computer. 
Readability formula programs could easily be tied Into the output, as 
could cloze-formatting programs. In addition, .records could be kept of 
the frequency of occurrence of technical terms (or other content terms) 
for use later in decisions on rewriting. In time, programs that provide 
alternative wordings (sets of synonyms, phrases, or even definitions) 
could be displayed for the writer at points likely to cause difficulty. 
But the W/'U/tGA would decide among them, noi the computer this the 

writer can do relatively easily and x^rell, but the computer cannot. Simi- 
larly, a computer program could be developed relatively rendily to flag 
difficult sentence constructions as long as it was not asked to transform 
such difficult cons tructicns . One of the suggestions to grow out of the 
Conference was that separating the writing and editing functions performed 
by a writer could make the development of computer assistance even more 
logical. For example, the computer could help the writer in his or her 
WKitLng role by storing and providing information useful in decisions con- 
cerning the organising and sequencing of content. On the other hand, the 
computer could help the writer in his or her tduting (rewriting) role by 
storing alternative wordings or flagging points of possible difficulty, 
i.e., "keeping track" (as noted above) of possible points of difficulty 
and reminding the writer of their existence as he goes ovor materials. 

Understanding and Improving 

Information Systems Within the Armed Services 

A persistent theme in Conference discussions was the practical pur- 
poses for which various written documents can and/or should be used, and 
the nature of the skills demanded of the readers in these uses. This 
appears to be partly an empirical question and partly a problem for sys- 
tems analysis. In order to understand the current information systems- 
within the Armed Services, a vigorous program is needed to determine the 
information demands in the various tasks that have to be carried out by 
a variety of occupational specialties, both on the job and in training 
to learu the job. It is also necessary to get reliable data as to hox-j 
these information needs are met at present, what skills these information 
sources require from users, and the attitudes of users as well as train- 
ing and command personnel toward the importance of requisite literacy 
and communications skills^ 

A prudent approach would be to supplement this important em.pirical 
work with a systematic analysis of what information systems ought to be 
like in order to be both efficient and effective. These analyses ought 
to consider le tasks that nVist be performed, as well as production, up- 
date, _and distribution problems, and the impact. of microfilm on technical 
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material production and use. It would probably be worthwhile to model 
and to simulate systems for various trade-offs between training and the 
use of perrormance aids. The problem of archival versus documentation for 
use deserves a closer examination for various practical circumstances. 
Manuals and accompanying materials specifically designed to teach were 
less emphasized at the Conference than manuals designed to facilitate job 
hrf°'^^;^r individual is on the job. The implication of this may 

be tna. the manuals are difficult to use in a training setting and force 
the instructor to improvise or that trainees are required to learn from 
manuals not especially designed for teaching them. Whether or not a di f- 
terentiation between job manuals on the one hand and teaching/ training 
manuals on the other. can make a significant improvement in -raining should 
oe a matter for study. 

The rapid and accelerating growth in the amount of technical publica- 
tions which are deemed necessary for hardware systems of increasing tech- 
nical sophistication suggest the need for a major re'-exan.ination of docu- 
mentation practices. Location of needed information, accessibility and 
updating, and physical bulk are only part of the problem. What appears 
to be also needed is better understanding of verbal and conceptual prep- 
ai^ation necessary to onerate p Ffp^ <-nirai->r „ ■ ■ r . 

Thpr-P oio ■ ^ operate ettectively withm a given information system. 

There are also indications that further developments in making text more 
readable and better suited to human purposes would not be fully utilif-ed 
unless the institutions and organizations which now produce technical doc- 
uments are modified. Concrete proposals for such organizational (and pro- 
cedural) changes did not emerge from the Conference, though there seemed 
to be some agreement that the promulgation of simple rules for judging the 
acceptability of documents was not sufficient even if these are incorpor- 
ated m regulations and procurement standards. 

Finally, an information system that is totally integrated from begin- 
ning to end could provide feedback to the people who decigu the hardware 
and software systems, so that they might consider the docume- tat^" on re- 
quirements (and resulting communications skills required of readers) at 
the same time that they make decisions about equipment 3tructure, function 
and maintenance. ' ^'-^"^'--^onj 
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.APPENDIX A 



BIOGRAPHICAL NOTES 
Biographical Notes on the P'resentors 



•JAMES R. BURKETT 

Ron Burkett is- Deputy Chief of the Instructional Technology Branch 
at the Technical Training Division of the Air Force Human Resources Lab- 
oratory. From 1968 to 1969 he was a Personnel Measurement Psychologist 
at the Personnel Research Division of AFHRL engaged in the development 
and revision of Air Force specialty knowledge tests. Since joining the 
Technical Training Division of AFHRL in 1969 as a Research Psychologist, 
ha has conducted and supervised research in the areas of instructional 
technology, readability, training evaluation, adult literacy skills, on- 
the-job training, and instructional systems and materials. Dr. Burkett 
has authored and co-authored over 20 technical reports, articles, and 
papers and was the 1974 winner of the AFHRL Donald B. Haines award for 
technical achievement. He received his PhD in Experimental Psychology 
from the University of Oklahoma. 



JOHN S. CAYLOR . . 

John Caylor is a Senior Scientist with the Human Resources Research 
Organization (HumRRO) . He has more than 13 years experience in research 
and development in military settings, including work in the determination 
of job reading requirements and research leading to a job functional lit- 
eracy training program. He has taught at the University of California, 
Riverside, and served as Evaluation Specialist for the Appalachian Adult 
Education Center. Dr. Caylor received his PhD from the University of 
Michigan. 



THOMAS E. CURRAN 



. Thomas Curran, of the Navy Personnel Research and Development Center 
m oan Diego, California, is Project Director for the Center's participa- 
tion m the Navy Technical Manual System research program. Previously 
he was Assistant Professor of Psychology at California State University, 
Uiico, and has served as an officer in the U.S. Navy. His ' prof essional 
specialties include the psychology of human learning, human development, 
and the psychology of language. Dr. Curran received his PhDin Education- 
al Psychology frpm the University of Minnesota. 
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THOMAS M- DUF'FY 



Torr. Duffy is a research psychologist at the Navy Personnel Re- 
search and Development Center. Prior to joining the Center .in 1973, 
he x^as an Assistant Professor in the Department of Psychology at the 
University of Manitoba, His current research involves the analysis 
of the reading skills of low literate adults and the development of 
programs for training functional literacy in the military context. 
Tnis work developed out of a career interest in' human memory and com- 
prehension processes and verbal learning. Publications have included 
investigation of imagery and verbal mnemonics in learning and memory, 
as, well as the requirements for reading skills in the Navy^ He re- 
ceived his PhD in 1^69 in experimental psychology from the University 
of Illinois. 



J. DEXTER FLETCHER ' 

Daxter Fletcher is a member of the Navy Personnel Research and 
Development Center. His principal interests -have been in the design, 
development, and evaluation of computer-assisted instruction with par- 
ticular emphasis on curriculum areas related to natural language proces- 
sing. Currently, he is -developing and evaluating techniques for the use 
of intelligent instructional systems for the Navy. Dr. Fletcher re- 
ceived an MS in computer science and a PhD~in educational psychology, 
both from Stanford University. , ' 



STEVEN D. GROFF . 

Steven Groff is a Captain in the U.S. Air Force and is assigned to 
the Technical Training Division of the Air Force Human Resources Labora- 
tocy. Prior to this assignment, he served as Technical Programs Monitor 
for technical training and flying training research at the Air Force Hum- 
an Resources Laboratory, Headquarters, Brooks AFE, Texas. He is currently 
engaged in research in the areas of literacy training and comprehensibilit 
measurement.. He received his MA in Experimental/Industrial Psychology frc 
the University of Tulsa in 1971. 
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JACK H. HILLER 



Jack Hiller is a Senior Project Director for the Army Res,^-irch 
Institute at the Presidio of Monterey, California. He is currently 
developing a reading comprehension test, and planning research to d.— 
ve^op a computer system which x^ould provide xrciters with ' diagnostic 
feedback. Previously, Dr. Hiller worked for the Southwest Regional 
-.a-- for Educational R&D, where he planned research in reading compre- 
hension, and designed instruction to promol.-. development of r.o-mprehen- 
sicn procesGes. He has also been an Assistant ^rofessor of Zducatiou- 
al Psychology at Southern Illinois University, Carbondale, Illinois, 
where he conducted extensive research in learning from text and lecture, 
and analysis of natural language data by computer. He has published 
numerous articles on using the computer to analyze text and transcribe 
speech and on verbal behavior and verbal learning,. He is a member of 
the .vmerxcan Psychological Association, American Educational Research 
Association, and is listed in ^o's ?/ho. Dr." Hiller earned h±3 PhD in 
educational Psychology f::om the University of Connecticut at Storrs 



ROBERT C. JOHNSON 



Robert Johnson is a research psychologist with the Air Force Human 
x.esources Laboratory. His work in job performance aids research includes 
development of handbooks detailing procedures for nroducing procedural- 
xj.ed technical data and a user acceptance and useability study of job 
guiae manuals. Prior to joining AFHKL., Mr. Johnson served as nn Air Force 
.n.aintenance officer, and later as a Human Factors Senior Engineer for the 
LocKheed-Georgia Company. He currently serves' as a maintenance officer 
ror an Onio Air National Guard fighter group. He holds a BS degree ."n 
.«ychoiogy from Northeast Missouri State University, and is currently 
enrniicd xn graduate work at che University cf Dayton. 

WlLLlAr4 G. MULLER , 

Kn!^.^'^''"' 1^"^^'^'' '-^ employed at the Naval Air Technical Services Facilif 
as tne Human Factor Specialist of the Naval Air Systems Command, Technical ' 
Documentation Policy and Prograins Office. In this position he directs 
and manages technxcal documentation research, development and evaluation, 
il.. Mulln?: also presently holds an additional position of Human Factor 
Consultant to the Naval Technical Manual System Program and as Human 
rnctors Advxsor to the NAVAIR Weapons Training Division (AIR-413) Prior 

Pa wSf ^^'■"'^ Development Center, Warminster, 

v^:' ^^^"^ P^^'"^^^ interests were in Technical Training and Human 
iactor.s Engxneering for Future Naval Weapon Systems. 
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RICHARD ?. KERN 



Richard Kern is a Senior Research Psycho locj.is l ai: the U.S. Amy 
Research Inotiuuce for 'che Behavioral ai:d Suci:il Sclencei5. Prior to 
this, he was -a Senior Research Scientist i-'ith Lhe Huina:^. Resources Re- 
search Organization. His major research j.iizeras'Ct, hav.-i been, in applied 
research on design of crainiu:^ and training laedia and in :^accors affect- 
ing quality of perf or:nance on the job. he is currently conducting re- • 
search on design of perf onaance-oriented nonresident ins truc::ional sys-- 
terns and their ^-.upportinr; materials and resea,'-.:h to ^ez-^lop job analysis 
raethodologles which will permit idenl* L.ficatio:: ot vj.-ii luni.iat ion ele- 
m'ents of job tasks and as.ses:-;ment of their related iL-.ki'. 1 req /'ireraents. 
He is a meinl^er of th'-i Auierican ?3ycholo;:;ical i'.;bsocj.o ; , r/ne /jnerican 
Educational Research As3ociation, and the T.nternation:r. Reac. in,?; Associa- 
tion. Dr. Kem recelvr-d his PhD fron; che S::.:iir.e Un.r.-er^-^ :.tv of Iowa. 



ARTHUR 1/ SIFGEL 

Arthur Siege] 1:^ D i.r.^'.:':or r,t Applied PsyciiO S^- trvi.ces , Inc. 
Previously he x.ras .-^--iw--::!-:,, /K^^i\oc [:\lc- a'. rerdVian Ua va: rr. :.;-y : ua.ic^rit: 
psychology at New York iinlversity^ at Q^ieens Co]Le^e. :-kvA at T.iMupie Uni- 
versity; and war, a Project Director at che lestitULe foi- Research in 
Huinan Reiati.oas. His studies have included rJv.:. de-t:; lojimen L of Tnethods 
for quantj.fyii-M; the un::eJ.lective and ^r.otor ln,^d :ne ^;ys i;e:a -operator , 
work method d-jve Loprien l , nianp:u;er plannir.^^; .an.t p».-:r.soane I allocation, 
perf ormani^e. a1 d development, traiuinf' s^'stem d<:.vo I opi;ie:\t , ■h'.iiiiavL relia- 
bility predicLiori, naintainability analysis, display ond control design, 
human transfer function analysis, aad system evalua'cior: - ' Vr . Seigel's 
e:-:perienc:e in linguistics and readability inclu..'es four Air "Force pro- 
grair.s, at least five Ka^/y prov^rains, and one Ariny program. Dr- Siegel 
has also developed a nuuiher of special techniques for analy^in^ and de- 
fining t>ie corr.prehenslbility of technical materials, he has published 
over 200 prof e;p5jional articles and technical rep-^rts, tv;o books, and has 
presented 3<^. papers before the American and as tern Psychological x\ssoci 
atzons. Dr. Siegel is a meTrl-)er of several profess j/.^nal assijeiations . 
He received the doctoral de.-^.ree from h'ew Vork Ur?..:-/ersl^ 
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JEAMNE S. CHALL 

Jeanne Chall is Professor of Education and Director of the Reading 

Laboratory, Graduate School of Education, Kar-vard University. She has . 

ducted research on readability, vocabulary, and .the psychology and teaching 
of reading. She is a Fellow in the American Psychological Association and " 
the American Association for the Advancement of Science, a past president of 
.the National Conference on Research in English, and is a member of the Board 
of the National Society for the Study of Education. Among her publications 
are the Dale-Chall Formula- for Predicting Readability (with Edgar Dale); 
Reada bilit y : An Appraisal of Research and Application ; Learning to Read: The 
Great Deb:-?te' and Toward a Literate Society , with John B. Carroll. Dr. Chall 
received her PhD fron the Ohio State Universicy. 

SAMUEL GIBBON 

Saniuel Gibbon has been with the Children's Television Workshop since 
1.963, when he joined that organization as one of the original Producers of 
S^esame Street. Previously, he was Associate Producer and VJriter of Captain 
Kangaroo for more than six years. In 1970 Mr', Gibbon undertook the develop- 
^^^^ °^ The Electric Company , a television program offering supplemental read- 
ing^instruction for second-^ third-, and fourth-graders experiencing reading 
difficulties, lie served as Executive Producer of that program until 1973. 
Since 1972 he has held an appointment as Lecturer in Education and Research 
Associate in the Center for Research in Children's Television at the Harvard 
University Graduate School of Education. His research interests focus on the 
effects of the properties of the television medium and the "syntax" of tele- 
vised messages on children's developing cognitive structures. Recent p. blica- 
tions include NATIONAL CONFERENCE ON VISU/vL INFORiMATION PROCESSING, (National 
Institute of Education, 1975); and "Sesame Street , The Electric Comp any, and 
Reading" (in J. Carroll and J. Chall, Eds., TOWARD A LITERATE SOCIETY, McGraw- 
Hill, Inc., 1975). Mr. Gibbon received his AB in English Literature from 
Princeton University. 
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ROBERT GLASER 



Robert Glaser is University Professor of Psychology & Education and 
Co-Director of the Learning Research & Development Center at the University 
of Pittsburgh. He has been President of the American Educational Research 
Association (1971-72) and of the Educational Psychology Division of the 
American Psychological Association. During 1969-70 Professor Glaser was a 
Fellow at the Center for Advanced Study in the Behavioral Sciences in Palo 
Alto, California. In 1970 he received the AERA / American Educational Pub- 
lishers Institute Award for Outstanding Educational Research in the fields 
of Instructional Materials. And, in 1973 he was elected to the National Aca- 
demy of Education. Professor Glaser has served as a consultant to many na- 
tional and international groups, including the Educational Subpanel of the 
President's Science. Advisory Committee. His recent research and publication 
is in the area of cognitive processes involved in' intellectual performance, 
and the design of school environments that are adaptive to individual differ- 
ences. Dr. Glaser received his PhD in 1949 in psychological measurement and 
experimental psychology from Indiana University. 

JOHN T. GUTHRIE 

John Guthrie has been the Director of Research" for the International 
Reading Association since September, 1974. Previously, he served as Kennedy 
Scholar in Education at the Kennedy Institute of the Johns Hopkins University 
Concept formation and verbal leaming^in children were his early research in- 
terests. In the past five years he has studied basic perceptual and langUcJge 
processes in reading, and in particular the cognitive deficiencies of poor 
readers. Aspects of Reading ' Acquisition , a volume recently edited by Guthrie 
presents a multidisciplinary perspective of learning to read. It will be pub 
lished by the Johns Hopkins University Press early in 1976. A second edited 
volume, Cognition, Curriculum, and Comprehension , which focuses on' theory and 
praccice in the development of reading comprehension., is in preparation. 
Dr. Guthrie received his PhD in educational psychology" from the University of 
Illinois. 
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GEORGE R. KLARE 

George Klare is Professor of Psychology al: Ohio University. Previously 
he^was Dean of the College of Arts & Sciences at Ohio University. Earlier 
affiliations were with the Training Research Laboratory of the Department of 
Psychology at the University of Illinois, and the Test Division of the Psycho- 
logical Corporation. Professor Klare has been a visiting scholar at Harvard 
University, State University of Mew York at Stonybrook, and in Europe as recip- 
ient of a Baker Award. He has served as consultant/lecturer for many organi- 
zations, and is listed in Mho's Who in America . Professor Klare's continuing 
research interests focus on readability and design of texts. His publications 
^"^1"^^ The Measurement of Readability (Iowa State Univetsity Press). Elementary 
Statistics - Data Analysis f or the Behavioral Sciences (McGraw-Hill) ', and Know 
Your Reader (Hermitage House). Dr. Klare received his PhD in psychology f7^ 
the University of Minnesota. 

MICttAEL MACDONALD-ROSS 

Michael Macdonald-Ross has been associated with the Ooen University in 
England since its start in 1969. From 1969 to 1971 he was' a partnc=:r in the 
educational consultancy. Instructional Systems Associates, which helped the 
University to set up its course development system. Since 1971 he has been 
Senior Lecturer in the University's Institute of Educational Technology. His 
research interests are diriicted coward curriculum design and the way print is 
used to communicate complex information. Currently, Mr. MacdonaLd-Ross is 
Director of the Institute's Textual Communication Research Group. This group 
conducts research inCo the i^ays students learn from texts, and advises the 
.University on ways to improve its printed teaching materials. Recenl publica- 
tions include papers on behavioral objectives and the structure of knowledge, 
a bibliography for textual communication, an analysis of Open University texts, 
and a review of research into typography. In addition, Mr. Macdonald-Ross is 
a Policy Editor of the Journal of Instructional Science and a Council Member of 
the Institute of Scientific and Technical Communicators. He received his BSc 
in zoology from University College, London. 

ERNST Z. ROTHKOPF 

Ernst Rothkopf is Head of the Learning and Instruction Research Department 
at Bell Laboratories. He joined the Bell research organization in 1958 after 
five years as a civ i.lian sciencis t in the Air Force's Personnel f. Training Re- 
search Center. Dr. Rothkopf was one of the first researchers in programmed 
instruction. His main interests throughout his career have been in learning 
tneory, training techniques, human verbal learning , and particularly, learning 
from written instructional material. He is the author of over sixty publica- 
tions and the editor of a book. Verbal Learning and the T e chnology of Written 
Instruction. Dr. Rothkopf has been a visiting or adjunct"prof essor at New York 
University, Rutgers , and the University of California at Berkeley. He is on 
several national advisory committees including the National Commission on Per- 
formance-Based Education, and is a Fellow of the American Psychological Associ- 
ation, as well as a member in numerous professional organisations. Dr. Rothkopf' 
received his PhD in experimental psychology from the University of Connecticut. 
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RICHARD L. VENEZKY 



since 



Richard Venezky has been on the. faculty at the University of Wisconsin 
2 1965, He has been a cember of the xNCTE Commission on Reading, the IRA 
Committee on Psycholiuguistics , and the Advisory Board of the Central Midwest 
Regional Laboratory's Language Program- He is also director pro tempore for 
the IRA Oral Language Staff Development Program. His current research includes 
ODservational studies of classroom organization (sponsored by the Israeli 
r-xnisnry of Education), analysis and review of teacher training techniques for 
oral language development (sponsored by the International Reading Association) , 
and diagnosis of prereading and early reading skills (sponsored by NIE) , His 
publications during the past year include monographs on testing and teachin^^ 
m reading, and a prereading skills program developed' at the Wisconsin Research 
and Development. Center. Professor Venezky is currently Professor and Chairman 

the Computer Sciences Department ,' and a principal investigator in the Wis- 
consin Research and Development Center for Cognitive, Learning. He received 
ais PhD in linguistics from Stanford University, 



THOMAS G. 5TICHT (Conference Chairperson) 

Tom SticHt is a Senior Staff Scientist with the Human Resources 
Research Organization/Western Division, Prior to this, he was a NASA 
Postdoctoral Fellow at the University of Pittsburgh, and Assistant Pro- 
fessor in the Department of Psychology at the University of Louisville-, 
In the Spring of 1975 he was Visiting Associate Professor at the Graduate 
School of Education, Harvard University. For spveral years. Dr. Sticht's 
research and development interests have centered on the functional literacy 
problems encountered by adults. This work has emphasized the improvement of 
readers through reading training, and the improvement of materials through 
design for readability and useability. Most of this work has been for Air 
Force, Army, and Navy sponsors, and has been brought together in two recent 
publications - Reading for Working: A Functional Literacy Anthology (HomRRO) , 
^'^'■^ Auding and Reading: A Developmental Model (HurcRRO WD^^.h others), 
Dr,. Sticht has been a consultant to local school districts, education labora- • 
tories, and government agencies, and is currently the Chairman of the Basic 
Education and Reading Committee of the International Reading Association. 
He received his PhD in experimental psychology from the University of Arizona, 
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APPENDIX B 



LIST OF CONFEREES: 

Conference on Reading and Readability 
Research in the Armed Services 



NAVy 



1. Dr. Marshall J. Farr, l^^cfebr 
Personnel and Training Research Programs 
Office of Naval Research (Code 458) 
Arlington, VA 22217 

0 

2. Dr. Edwin Aiken 

Nav>- Personnel Research and Development Center (Code 306) 
San Diego, CA 92152 

3. CDR Larry Beguin 
Bureau of Naval Personnel 
PERS-2S . . 
Washington, DC 20320 

4.. Dr- Thomas Curran 

Navy Personnel Research and Development Center (Code 306) 
San Diego, CA 92152 ^ 

5. Dr. Thomas Duffy 

Navy Personnel Research and Development Center (Code 306) 
San Diego, CA 92152, ^ 

6. Dr. Dexter Fletcher 

Navy Pers9nnel Research and Development Center (Code 306) 
San Djiego, CA 92152 ^ 

7. Dr. Eugene E. Gloye 
Office of Naval Research 
Pasadena Branch Office 
1030 East Green Street 
Pasadena, CA 91101 

8. Dr. Norman J. Kerr 
Head, Research Section 

S^.r'^f ^^""^^fl ^"^^1^^*^^°^ Research Branch (Code 341) 

Chief of Naval Technical Training ^ 
NAS Memphis (75). 
Milling ton, TN 38054 

9. Mr. William Muller 

NAVAIR Technical Services Facility 
NATSF . ^ 

700 Robbins Avenue 

Philadelphia, PA 19111 * 3 1 1 
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AIR FORCE 



ICX, Dr. James R, Burkett 

Air Force Human Resources Laboratory 
Technical Training Division 
Lowry Air Forca Base, CO 80230 

11. Dr. Ronald P, Carver 

University of Missouri at Kansas City 
Kansas City, MO 64110 

12. CAPT Charles Curran . 
ATC/XPTT 

P- rrlolph'AFB, TX 78148 

13. CAPT Steven G. Groff 

Air Force Human Resources Laboratory 
Technical Training Division 
Lowry AFB, CO 80230 

14. Dr. Raymond L- Le^^;iski 
Extension Course Institute 
Educational Systems Branch 
Gunter AFB, AL 36114 

15. Dr. Arthur I- Siegel 
Applied Psychological Services 

■ 404 E. Lancaster Avenue 



U.S. Army Research Institute Field Unit 
P. 0. Box 5787 

Presidio of Monterey, CA 93940 

18, Mr. R. T. Diduk . 
HQ TRADOC (ATPR) 
DCS Personnel 

■Education Branch ^ 
. Fort Monroe, VA 23651 



Wayne, PA 19087 



ARMY 




17. Dr. Richard D. Bloom 
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19. 1-Ir. Dennis F. Fisher 
Behavioral Resiearch Directorate 

US Army Human Engineering Laboratory 
Aberdeen Proving Ground, MD 21Q05 

20. Mr. George Fry 
Education Specialist 
Army-Wide Training Literature 
•U.S. Army Signal School 
Tort Gordon, GA 30905 

21. l(jAJ William Highlander ^ 
HQJRADOC (ATTNG-AS) 

Collective Training Branch, DCST " 
Literature Production Section 
Fort Monroe, VA 23651 

22. ' Dr. Jack H.. Killer 

US Army Research Institute Field Unit 
P.O. Box 5787 

Presidio of Monterey, CA 93940 

23. Dr, Otto Kahn 

US A.rmy Research Institute Field Unit 
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